It's possible to skip any wizzrobe mid-fight cutscene by damaging the wizzrobe during its 2 frames of vulnerability right before the cutscene begins, and doing so enough times to kill the wizzrobe. However, this can lead to a softlock with the Secret Shrine wizzrobe specifically.
Punch the wizzrobe on the first two cycles, being careful not to get too close too soon and make it disappear. To skip the mid-fight cutscene, time a pound to begin when the wizzrobe's head has just hit its leftmost point. Continue holding A after pounding the ground to keep the pound's hitbox active as long as possible. Then back away slightly and begin a full combo right after the wizzrobe fully disappears into the portal.