I played Y4 and not Y5, but chapter 1 in Y6 to me felt like it picked up after Y4 and Y5 was a filler episode or something. Like all the dialog in chapter 1 just felt like a time skip happened between seasons in an anime series
Not quite sure where you got this impression. The entire first 40 or so minutes of the game is a retelling of the ending of Yakuza 5 and the events that come after for Haruka, Kiryu and co are the direct results of the events happening in Yakuza 5 for example
Kiryu, Seijima going to prison for the events that took place in 5, Haruka running away from the orphanage due to the all the negative attacks she's getting from the press after revealing she's Kiryu's "daughter" at the end of 5. Kiryu returning to the orphans as well is something picked up from Yakuza 5 as he had been away for some time due to leaving the orphanage in Yakuza 5. Akiyama is in hiding because the Chinese mafia is coming after him due to his relation to Kiryu and the Tojo Clan - again due to his part in Yakuza 5.