Machine learning model calculates chemotherapy success in patients with osteosarcoma

A research team at Johns Hopkins has created and trained a machine learning model to calculate percent necrosis (PN) — or, what percentage of a tumor is “dead” and no longer active — in patients with osteosarcoma, a type of bone cancer. The model’s calculation was 85% correct when compared to the results of a musculoskeletal pathologist. Upon removing one outlier, the accuracy rose to 99%.

A post-chemotherapy PN calculation helps provide the patient with a prognosis for survival. For example, a PN of 99% indicates that 99% of the tumor is dead, suggesting chemotherapy was effective and the patient has improved odds of surviving. Pathologists calculate PN by looking at, interpreting and annotating whole-slide images (WSIs), which are thinly sliced sections of a specimen (bone tissue, in this context) that are mounted onto slides for microscopic analysis.

“Calculating the PN is a labor-intensive process that requires a lot of annotation data from the musculoskeletal pathologist,” says Christa LiBrizzi, M.D., co-first author of the study and a resident with Johns Hopkins Medicine’s Department of Orthopaedic Surgery. “Additionally, it has low interobserver reliability, meaning that two pathologists trying to calculate a PN from the same WSIs will often report different conclusions. Due to these factors, we thought trying to calculate a PN by alternate means was a worthwhile effort.”

The team sought to develop a “weakly supervised” machine learning model, one that required minimal annotation data to be trained on. Training the model this way would mean that a musculoskeletal pathologist using the model to calculate a patient’s PN would only need to provide it with partially annotated WSIs, thus reducing the pathologist’s labor burden.

First, the team gathered data, including WSIs, from the pathology archives of Johns Hopkins’ U.S. tertiary cancer center. All data came from patients with intramedullary osteosarcoma — that is, osteosarcoma that originated in the center of the bone — who underwent chemotherapy and surgery at the center between 2011 and 2021. The team then had a musculoskeletal pathologist partially annotate three types of tissue on each of the gathered WSIs: active tumor, dead tumor and non-tumor tissue. The pathologist also estimated the PN for each patient. Using this information, the team began to train the model.

“We decided to train the model by teaching it to recognize image patterns,” says Zhenzhen Wang, co-first author of the study and a doctoral student in the Johns Hopkins Department of Biomedical Engineering. “We segregated the WSIs into thousands of small patches, then divided the patches into groups based on how they were labeled by the pathologist. Finally, we fed these grouped patches into the model to train it. We thought this would give the model a more robust frame of reference than simply feeding it one large WSI and risking missing the forest for the trees.”

After being trained, the model and the musculoskeletal pathologist were given six WSIs to interpret from two osteosarcoma patients. Results showed an 85% positive correlation between the model and the pathologist’s PN calculations and tissue labeling. The model did not always properly label cartilage, which led to an outlier due to an abundance of cartilage on one WSI. When the outlier was removed, the correlation increased to 99%.

“If this model were to be validated and produced, it could help expediate the evaluation of chemotherapy’s effectiveness on a patient — and thus, get them a prognosis estimate sooner,” says LiBrizzi. “That would reduce health care costs, as well as labor burdens on musculoskeletal pathologists.”

In future studies, the team aims to include cartilage tissue in the model’s training and to diversify the WSIs to include other types of osteosarcoma beyond intramedullary.

The study was published online Oct. 5 in the Journal of Orthopaedic Research. Additional Hopkins authors include Jeremias Sulam, assistant professor of biomedical engineering; Aaron W. James, professor of pathology; and Adam S. Levin, assistant professor of orthopaedic oncology. Carol D. Morris, chief of the orthopaedic surgery service at Memorial Sloan Kettering Cancer Center, also contributed to the study.