Simple code to do clustering and SSE plot

[vcellmike]

Consider inverse_features code and strip it from everything except importing data frame, clustering, and plotting SSE. Test it for several small dataframes.

Outputs: one dataframe per number of clusters.

Don't add inverse features - it should be a separate function that expands the data frame.

[timothykuliyev]

Progress:

• implemented a stripped-down version of Inverse-features that only creates an elbow graph
• implemented cut switch and cut_number variables. Cut can be found in the lines after the import of the pickle file. Setting cut to True, means the resultant dataframe will output a smaller one cut down to the number of rows set in cut_number.
• The Kmeans graph has clusters up to 15. The elbow/sudden drop occurs at ~6 clusters.
• The OUTPUT: cluster_dicts dictionary. You may set num_clusters to any number. The kmeans SSE clustering will cluster the dataframe by num_clusters and append to the dictionary. eg. num_clusters = 3, outputs three dataframes like this:

{"1": dataframe, "2": dataframe, "3": dataframe}

Notes:

• PCA Code is identical in inverse_features and PCA notebook
• It appears that the inverse features are already implemented in the original pickle files

[timothykuliyev]

Further progress:

• Created a function that returns a cluster DataFrame. Inputs are: Num_clusters, clustering_column_name (use to determine what the column name of the cluster column will be).
• One can use this function to return a pandas DataFrame which can then be saved using .to_pickle() to an appropriate directory.
• This function provides a very quick way of doing clustering.