It's about 5.5 hours for us as well. What we've done usually, is to gather everything into one spot in advance. Then, pack the truck first thing in the morning on the day that we're going, and make the long drive starting before lunch. That we we can get there with plenty of daylight, set up the tent, then look for something to have for dinner. By doing the return trip in a similar fashion, the two travel days are quite manageable, and still have some time left over to do things on site.
If you go to the event for 5+ days total, setting aside two of the days as mostly travel isn't so bad.