I also agree that VI has the best music overall. It really is the most "symphonic" one, with a harmonious structural thematic recurrence, while also combining the baroque counterpoint of the Alefgard trilogy and the more Romantic tonalities of V.
IV really marks for me Sugiyama's experimental period, when he breaks away from his Bachist technique and into more modern composition.
(By that I meant the different recording with the London Philharmonic, Tokyo Philharmonic and NHK Radio Orchestra. I prefer the London one...but it's also one of the most prestigious ensembles in the world.)