1 My datasets

1.1 Private datasets

ReProMSig allows users to upload unpublished datasets for development and validation of prognosis models. The uploaded dataset can be used for Signature Development or Single Patient Prediction analysis.

• A patient annotation file (clinicopathological parameters) is required to be provided as minimal input for the analysis (Figure 1a). If a molecular profile file is provided too, it can be used together with patient annotation file in the analysis at user's will.
• Molecular assay platform and Log base are only needed if a Molecular profile file is provided.
• Once a dataset is uploaded, key information and statistics (frequency of each uploaded clinicopathological parameter) will be shown at the "Uploaded user datasets" section (Figure 1b)

1.1.1 Molecular profiles (File format)

Molecular profile file is a feature-by-sample matrix in TXT/CSV/Excel format. Feature IDs should be provided at the first column with a column name "ID". The additional columns are molecular features for each sample with sample identifiers as column names. Molecular features could be gene mutation status, mRNA/non-coding RNA/protein quantification levels, methylation levels, etc. Figure 2 is an example of mRNA expression levels profiled by microarray.

1.1.2 Patient annotation (File format)

Patient annotation file consists of three groups of columns including fixed columns, endpoint columns and custom columns. The names of fixed columns should be identical to the table template file shown in Figure 3. These clinicopathological parameters will be used for query samples suitable for analysis, as well as for prognosis model development. Please note that missing values in all columns should be provided as "NA".

Fixed columns:

• Patient_ID needs to be a unique patient identifier. Any ID that can be linked with personal identity is not permitted, such as national card ID, social security ID, passport ID.
• Sample_ID needs to be a unique sample identifier.
• Sample_type: "Primary tumor", "Metastatic tumor", "Recurrent tumor".
• Is_primary_treatment indicates whether the provided treatment info (Treatment_type, Treatment_settting, Regimen) is the primary treatment following the initial diagnosis, including "Yes" or "No".
• Treatment_type indicates the type of treatment regimen that a patient received, including "Ablation", "Bone marrow transplant", "Chemotherapy", "Endoscopic therapy", "Hormonal therapy", "Immunotherapy", "Photodynamic therapy", "Radiation therapy", "Stem cell transplant", "Surgery", "Targeting molecular therapy".
  Note 1: To include multiple values, separate values need to be concatenated by '+' as separator.
  Note 2: If Treatment_type is "Surgery", values of Treatment_setting and Regimen columns need to be "Not applicable".
• Treatment_setting describes the treatment's purpose in relation to the primary treatment, including "Adjuvant", "Conditioning", "Consolidation", "Induction", "Locally advanced", "Maintenance", "Metastatic", "Mobilization", "Neo-adjuvant", "Not applicable", "Preventative", "Radiosensitization", "Salvage".
  Note: To include multiple values, separate values need to be concatenated by '+' as separator.
• Regimen refers to regimen that a patient received.
• Gender: "Female" or "Male".
• Ethnicity: "Asian", "Black or African American", "Unknown", and "White".
• T/N/M columns stand for Tumor (T), Nodes (N), and Metastases (M) stage, which are formatted as "T/N/M & Arabic numerals", e.g. "T1", "N0", or "M0".
• Stage_edition indicates the edition of AJCC stage, e.g. "AJCC 6th", "AJCC 7th" or "AJCC 8th".
• Stage refers to AJCC stage, which includes "I", "II", "III" or "IV".

Endpoint columns can be one or more of "OS" (overall survival), "DFS" (disease-free survival), "RFS" (recurrence-free survival), "DSS" (disease-specific survival), "PFS" (progression-free survival), and "TTP" (time to progress) depending on the availability.

• The column name of survival time is a combination of Endpoint type and Survival time unit with a separator "_", e.g. "OS_months", or "OS_days".
• The column name of survival status starts with endpoint type followed by "_status", e.g. "OS_status".

Custom columns: Users can append custom columns after the fixed and endpoint columns, e.g. Lauren classification, EBV status, or molecular subtype for gastric cancer studies.

Note: space in column names will be automatically replaced by underscore sign.

1.2 Public datasets

We curated multiple published datasets to be integrated in the ReProMSig (Figure 4), to allow re-analysis, meta-analysis, as well as a quick test of the system. Users can select a dataset and view details (Figure 5) by clicking "Dataset details" button. These public datasets may be used as training or validation sets, as they have been carefully curated.

2 My signatures

2.1 Private signatures

Developed signatures by Signature development module will be automatically saved as private signatures for user management (Figure 6).

• Clicking "Signature details" button will show the details in development, validation and evaluation procedure of the selected signature.
• Clicking "Patient prediction" button will quickly load Single Patient Prediction module with the selected signature.
• Clicking "Reporting file" button will show the generated signature report file, which is structured following the TRIPOD guideline with full details of methods and results in developing and validating a signature model.
• Clicking "Delete" button will remove the selected signature from the system.
• Clicking "Local upload" button will allow users to upload local signature model and reporting file that were generated using the standalone version of ReProMSig (https://github.com/WuOmicsLab/ReProMSig).

2.2 Public signatures

We provide a published prognostic signature (ColoGuidePro), reproduced by ReProMSig (Figure 7) based on signature parameters from the publication, and new signatures developed by ReProMSig for stage II (ColoGuide_Stage_II-Msig) and stage III (ColoGuide_Stage_III-Msig) colorectal cancer patients respectively. Users can quickly access the "Single Patient Prediction" module by clicking "Patient prediction" button with a selected public signature.

2.3 Public signature reports

Users can share the signature reporting file to public or keep it private at own's will. A published signature report (Figure 8) is visible to all guest or registered users.

• Clicking "Reporting file" button will show the signature reporting file of the selected signature.
• Each report file has a unique ID and link to the server to check the integrity of the report.

3 Signature development

3.1 Search datasets

Users can specify a combination of specific clinicopathological parameters (Figure 9) to search samples of interest, which will be used as training or validation datasets in signature development. The details of parameters are described at Private datasets section.

Please note that variables with * sign allow multiple selection.

• For predictive signature (Figure 10), users can define control group (standard treatment) and treatment group (treatment of interest) by specifying corresponding "Treatment type", "Treatment setting", and "Regimen"

3.2 Select training and validation datasets

The statistics of samples meeting the query criteria will be summarized for each dataset and listed in an interactive table (Figure 11a).

• In the "Number of patients found" column, the number of patients meeting the query criteria and the number of all patients in this dataset are separated by a slash.
• In the dataset selection area (Figure 11b), users need to choose a dataset used for model training, and specify one or more datasets used for external validation, which dropdown list allows multiple selection.
• If expression profiles were provided in a dataset, density plots will be generated to present log2-transformed expression distributions for all queried datasets (Figure 11c).

3.3 Parameters for signature development & validation

Users can specify the critical parameters (Figure 12) in the procedure of development, validation and evaluation of prognosis model. Details of each parameter will be shown by clicking the nearby question mark icon.

3.4 Results

The results of signature development, validation and evaluation are categorized in multiple tabs as below:

• Summary tab shows summary of the developed signature, clinicopathological statistics, molecular profiles distribution for used training dataset and validation dataset(s). (Figure 13)
• Signature details tab shows the details of the developed prediction model, including predictors, regression coefficients, hazard ratios (HRs) accompanied by 95% confidence interval, and P values. For expression signature, the Pearson correlation between each pair of the signature predictors is plotted for visual check if their expression levels are independent. (Figure 14)
• Signature performance tab introduces the performance of the developed signature from multiple perspectives. (Figure 15)

• For prognostic molecular signature, independence test is used to evaluate the independence of the developed molecular signature, compared with and the clinicopathological variables in the training dataset.
• For predictive molecular signature, univariate and multivariate Cox regression analyses were performed to test for interaction between treatment response and the predicted treatment-benefit groups in terms of survival, in the training dataset.
• Kaplan-Meier method is applied to assess the difference of survival between different patient groups.
• The time-dependent receiver operating characteristic (ROC) at specified intervals (e.g. 1, 3, and 5-year) is generated to quantify and compare the discrimination ability of the molecular signature and clinicopathological predictors in the training dataset.
• The calibration plots at specified intervals (e.g. 1, 3, and 5-year) are generated to assess the calibration performance of the prognostic/predictive molecular signatures by comparing predicted probabilities and observed outcome.
• The prediction error (PE) curve is generated to evaluate overall performance of the developed model in the training dataset.

• Clinicopathological association tab: Association analysis is applied to evaluate the correlation between the developed prognosis model and clinicopathological variables in the training and validation datasets respectively. (Figure 16)
• Nomogram tab shows the details of tools helping clinical decision, based on the developed model. (Figure 17)

• Nomogram is generated for predicting event probabilities at specified intervals (e.g. 1, 3, and 5-year) of a new case, which may serve as a tool facilitating clinical decision.
• The calibration plots at specified intervals (e.g. 1, 3, and 5-year) are generated to assess the calibration performance of the Nomogram by comparing predicted probabilities and observed outcome.

• TRIPOD report is available in "My signatures" module by clicking "Reporting file" button.

3.5 Reporting file

The full details of methods and results in developing and validating a signature model are presented as a single HTML file with structure following the TRIPOD guideline check items (items 4-16).

• The reporting file (Figure 18) can be edited, downloaded and shared to public in "My signatures" module.
• The report items include editable items and locked ones. Users are required to provide the information or edit the template text for editable items, including study design, eligibility criteria for participants, treatment details, etc.
• Note: The uploaded local reporting file (Figure 19) can also be shared to public in "My signatures" module. The admin will check the integrity of the uploaded signature model and the reporting file and notify the results by email within 48 hours.
• The shared reporting files of three signatures for colorectal cancer prognosis are listed below as examples:

• ColoGuidePro: https://omics.bjcancer.org/prognosis/open/ColoGuidePro_4ec64bfa_report.html
• ColoGuide_Stage_II-Msig: https://omics.bjcancer.org/prognosis/open/ColoGuide_Stage_II-Msig_a1cffaed_report.html
• ColoGuide_Stage_III-Msig: https://omics.bjcancer.org/prognosis/open/ColoGuide_Stage_III-Msig_4e0d91d3_report.html

4 Single Patient Prediction

4.1 Input data

• Firstly, users need to visit the My signatures module, select a signature and click the "Patient prediction" button. Then signature details and the used training dataset will be shown (Figure 20a).
• For molecular signatures, users need to upload molecular profiles with all molecular features utilized in the selected signature (highlighted in Figure 20b).
• For traditional signature, users need to provide value of each clinicopathological predictor utilized in the model (i.e. Nomogram).

4.2 Select cohorts

Datasets suitable for the selected signature are listed in the summary table (Figure 21) for selection, which will be used as reference cohort(s) for indicating ranks of predicted risk score (RS) of this case.

4.3 Results

The output of risk prediction for an individual is shown as below:

• The risk score, predicted risk group (or treatment-benefit group for predictive signature), and specified event probabilities (e.g. 1, 3, and 5-year) of this case are described in Figure 22a.
• An risk score (RS) ranking feature (Figure 22b) is provided to indicate the percentile rank of predicted RS for this individual, in the RS distribution of each selected dataset (with the same clinical endpoint and all utilized predictors) (Figure 22c).