This GitLab repository contains the main material used in the paper [title] by [authors].
This GitLab repository contains the main material used in the paper [title] by [authors].
The study examines patients undergoing treatment for alcohol disorders, utilizing machine learning techniques to predict clinical success or withdrawal. The main goal is to employ explainability tools to assess the impact of individual versus social factors on treatment outcomes. Additionally, the research explores whether the significance of these factors changed during the pandemic by comparing pre-pandemic and post-pandemic patient groups.
[Impact?]
## About the Dataset
[Origin, Characteristics]
The dataset has not been provided since the authors do not have permission for its sharing from the data providers.
## Dealing with Class Imbalance
One of the primary challenges we encountered was a significant class imbalance, with a higher number of patients withdrawing from treatment compared to those staying.
To address this issue, we implemented four different training approaches or pipelines on both the pre-pandemic and post-pandemic training datasets:
1.**Using the Original Dataset (ORIG)**: The models were trained on the original datasets.
2.**Class Weight Adjustment (ORIG_CW)**: The models were trained on the original datasets but were penalized more heavily for misclassifying the minority class.
3.**Oversampling (OVER)**: Additional samples were generated for the minority class (patients staying) to balance the dataset.
...
...
@@ -24,32 +28,36 @@ To address this issue, we implemented four different training approaches or pipe
These approaches resulted in multiple training datasets. However, to ensure a fair comparison of the models' performance across different pipelines, we utilized a common test dataset for evaluation, irrespective of the training approach followed.
*[output](./EDA/output): Plots about feature distributions, correlations and importance.
*[EDA.ipynb](./EDA/EDA.ipynb): Exploring and filtering data, handling missing values, encoding variables, building the final pre- and post- pandemic datasets, and generating plots for feature distributions, correlations and importance.
*[gen_train_data](./gen_train_data):
*[gen_train_data.ipynb](./gen_train_data/gen_train_data.ipynb): Generating training and testing datasets for each of the pipelines.
*[model_selection](./model_selection):
*[hyperparam_tuning.py](./model_selection/hyperparam_tuning.py): Tuning models through a random search of hyperparameters.
*[cv_metric_gen.py](./model_selection/cv_metric_gen.py): Generating cross-validation metrics and plots for each of the tuned models.
*[cv_metrics_distr.py](./model_selection/cv_metrics_distr.py): Generating boxplots for each cross-validation metric and tuned model.
*[test_models.py](./model_selection/test_models.py): Testing tuned models with test dataset.
*[fit_final_models.py](./model_selection/fit_final_models.py): Saving fitted model for each selected final model.
*[output](./model_selection/output):
*[hyperparam](./model_selection/output/hyperparam): Excel file containing the optimal hyperparameters for each model in each pipeline.
*[cv_metrics](./model_selection/output/cv_metrics): Material related to the results of cross-validation: scores, ROC and Precision-Recall curves and boxplots for each metric.
*[testing](./model_selection/output/testing): Material related to the results of testing the tuned models: scores, ROC and Precision-Recall curves and confusion matrices.
*[fitted_models](./model_selection/output/fitted_models): Final selected trained models.
*[EDA](./EDA):
*[results](./EDA/results): Plots about feature distributions, correlations and importance.
*[EDA.ipynb](./EDA/EDA.ipynb): Exploring and filtering data, handling missing values, encoding variables, building the final pre- and post- pandemic datasets, and generating plots for feature distributions, correlations and importance.
*[gen_train_data](./gen_train_data):
*[gen_train_data.ipynb](./gen_train_data/gen_train_data.ipynb): Generating training and testing datasets for each of the pipelines.
*[model_selection](./model_selection):
*[hyperparam_tuning.py](./model_selection/hyperparam_tuning.py): Tuning models through a random search of hyperparameters.
*[cv_metric_gen.py](./model_selection/cv_metric_gen.py): Generating cross-validation metrics and plots for each of the tuned models.
*[cv_metrics_distr.py](./model_selection/cv_metrics_distr.py): Generating boxplots for each cross-validation metric and tuned model.
*[test_models.py](./model_selection/test_models.py): Testing tuned models with test dataset.
*[fit_final_models.py](./model_selection/fit_final_models.py): Saving fitted model for each selected final model.
*[results](./model_selection/results):
*[hyperparam](./model_selection/output/hyperparam): Excel file containing the optimal hyperparameters for each model in each pipeline.
*[cv_metrics](./model_selection/output/cv_metrics): Material related to the results of cross-validation: scores, ROC and Precision-Recall curves and boxplots for each metric.
*[testing](./model_selection/output/testing): Material related to the results of testing the tuned models: scores, ROC and Precision-Recall curves and confusion matrices.
*[fitted_models](./model_selection/output/fitted_models): Final selected trained models.
*[explainability](./explainability):
*[compute_shap_vals.py](./explainability/compute_shap_vals.py): Computing SHAP values for final models.
*[compute_shap_inter_vals.py](./explainability/compute_shap_inter_vals.py): Computing SHAP interaction values for final models.
*[shap_plots.py](./explainability/shap_plots.py): Generating SHAP summary plots for the SHAP and SHAP interaction values computed. Comparing major differences between pre- and post-pandemic groups.
*[output](./explainability/output): SHAP and SHAP interaction summary plots.
*[compute_shap_vals.py](./explainability/compute_shap_vals.py): Computing SHAP values for final models.
*[compute_shap_inter_vals.py](./explainability/compute_shap_inter_vals.py): Computing SHAP interaction values for final models.
*[shap_plots.py](./explainability/shap_plots.py): Generating SHAP summary plots for the SHAP and SHAP interaction values computed. Comparing major differences between pre- and post-pandemic groups.
*[results](./explainability/results): SHAP and SHAP interaction summary plots.
