Remove the need for always pushing the Python folder for x64/x86 for every new version

[ader1990]

The reasons are multiple:

• unreviewable PRs - this kind of PR has 300K+ changed files
• chain of trust becomes impossible to track (from the installer to the actual folder content)

Options available:

• add a Python installer URL and checksum parameters
  This also needs a way to automate the installation to the current directory (already available in the Readme) and a way to make sure that there was no previous installation of the same py version, plus cleanup (uninstall) as try/finally step
• investigate the option for bit-reproducible builds for the C extensions in Python

[petrutlucian94]

Another idea would be to just bundle the Python MSI into the cloudbase-init msi. We could fetch it at build time instead of having it part of the repo.

There are two possible approaches:

• msm packages: https://learn.microsoft.com/en-us/windows/win32/msi/about-merge-modules
• msi bundles: https://learn.microsoft.com/en-us/windows/msix/package/bundling-overview

[petrutlucian94]

Then there's the embeddable archive that we could fetch at build time: https://docs.python.org/3.7/using/windows.html#the-embeddable-package

[claudiubelu]

Interestingly, we seem to have had this idea in the past for Python 3.4, this file still exists: https://github.com/cloudbase/cloudbase-init-installer/blob/00dbebcc368e8bafb07b2f47a6043055f78ce0f2/BuildAutomation/SetupPython34Template.ps1 . I wonder why this idea was given up on. It may be because some dependencies are harder to build / install / source this way? That used to be the case, some things were a pain to build / install for some architectures on Windows.

The commit seems to suggest that; 364 days later, the Python 3.4 template was added, followed several commits amending the template by adding various DLLs and packages. It's hard to tell if those updates have been kept for future Python versions (I assume not). And there doesn't seem to be any commits like that for the latest Python versions added, though, as you mentioned, the commits are too large to even tell if there's anything different from a vanilla Python installation. So, the question is, is there anything different we're doing to those templates? If not, I'm for having a build script / bundling the Python MSI into cloudbase-init.

[ader1990]

Interestingly, we seem to have had this idea in the past for Python 3.4, this file still exists: https://github.com/cloudbase/cloudbase-init-installer/blob/00dbebcc368e8bafb07b2f47a6043055f78ce0f2/BuildAutomation/SetupPython34Template.ps1 . I wonder why this idea was given up on. It may be because some dependencies are harder to build / install / source this way? That used to be the case, some things were a pain to build / install for some architectures on Windows.

The commit seems to suggest that; 364 days later, the Python 3.4 template was added, followed several commits amending the template by adding various DLLs and packages. It's hard to tell if those updates have been kept for future Python versions (I assume not). And there doesn't seem to be any commits like that for the latest Python versions added, though, as you mentioned, the commits are too large to even tell if there's anything different from a vanilla Python installation. So, the question is, is there anything different we're doing to those templates? If not, I'm for having a build script / bundling the Python MSI into cloudbase-init.

that code was not relevant anymore, as Python changed the packaging scheme.

Opened a draft PR here: #46

[ader1990]

Then there's the embeddable archive that we could fetch at build time: https://docs.python.org/3.7/using/windows.html#the-embeddable-package

This option has been tried and it works as long as there is no need to build cpython C extensions. But we need alot of those to build during cloudbase-init package build. The option for this to work would be to have all the pip freeze packages already built as wheels, and then just go over them one by one and install them. The problem is that the openstack requirements / upstreams are always changing.