Weihua Zhou

GRAC represents a surgical scene as a graph, retrieves graph-caption pairs of similar surgical activities from the training set, and compares instrument-tissue interactions between the current and retrieved scenes to condition caption generation. On the combined EndoVis-18 benchmark it reaches a CIDEr-D of 5.88, against 4.23 for SGT and 3.36 for the strongest conventional Transformer baseline, and an exact instrument-action-target triplet-set accuracy of 52.6%, against 32.7% for SGT and 26.2% for the strongest conventional baseline. On TORS it reaches a CIDEr-D of 5.06, against 3.80 for the strongest baseline.

GRAC represents a surgical scene as a graph, retrieves graph-caption pairs of similar surgical activities from the training set, and compares instrument-tissue interactions between the current and retrieved scenes to condition caption generation. On the combined EndoVis-18 benchmark it reaches a CIDEr-D of 5.88, against 4.23 for SGT and 3.36 for the strongest conventional Transformer baseline, and an exact instrument-action-target triplet-set accuracy of 52.6%, against 32.7% for SGT and 26.2% for the strongest conventional baseline. On TORS it reaches a CIDEr-D of 5.06, against 3.80 for the strongest baseline.