Integrate Fetch Scripts

[IsaacMilarky]

Integrate Fetch Scripts

Problem

Currently the scripts don't work well with each other to merge all of our data from various sources.

Solution

Create a super script that calls both scripts and merges the indexes as well as the data itself.

Result

Summary:

• Create __init__.py file to make python recognize scripts folder as a module
• add pull-all-assets.py file to make scripts be able to be called from one invocation
• Move the logic that merges two indexes to a separate file in a function for reuse
• Use this method in pull_public_assets.py to merge legacy index with the newly generated one
• Rename files to use underscores since you can't import files with dashes in them in Python
• Add a function to determine which codejson file is the most up to date and then return the most up to date one
• Use that function so that only the most up to date codejson file is saved in each script

Test Plan

Run the script locally

[sachin-panayil]

LGTM! 🔥