Yamabushi didn't just know ninjitsu, they hired ninjas do due fighting for them.
"In addition to their spiritual or mystical abilities, yamabushi are often attributed with being skilled practitioners of ninjutsu, the art of the ninja. The mountain monks are known to have hired ninja to fight alongside them, and to aid them in various, more clandestine ways. And the ninja are known to have disguised themselves as monks or mountain ascetics, so as to pass unnoticed more easily in certain environments. Most likely, this is where the confusion can be said to have come from; it seems unlikely that any significant number of yamabushi would have been trained by the insular ninja clans in ninjutsu.
In fact, the ninja clans were a derivation from the yamabushi lifestyle, merged with different interpretations of ninpo (the highest form of ninjutsu) and embraced by the common people."