What is the most challenging but rewarding system have you learned?
Language.
See, it’s both a serious answer and a joke because the question is grammatically incorrect.
But relevant to gaming, I’d say D&D, mostly because it’s a complex system and the first I learned. By the time I started into other systems I had already cut my teeth on tabletop gaming, so learning smaller, simpler sets of rules didn’t bother me. And the reward technically includes every session I’ve ever run, so that’s too vast to ignore. Every other system I’ve played, even counting those outside tabletop gaming, both was easier to learn and gave me less return on my investment in terms of game time, social interaction, and emotional satisfaction. It’ll be a good while before something comes close to usurping it.