Not when you watch the episodes, there are very specific references to what happened in Swan Song in later episodes, I can remember exactly which one but either 5, 6 or 7.
I just noticed a screw up in the order of the episodes in season 5 of Castle.
Specifically regarding Ep04 Swan Song. It's listed as Ep07 with MyEpis ...