Nobody thinks the flag only represents racism. Of course it is a part of history, but when it is still flying on a government building and was set there for the soul purpose of opposing the civil rights movement, there shouldn't be much question in your mind as to its legitimacy. I don't support a ban on the confederate flag, and these companies like Apple who are removing it from the app store are being silly - clearly just to appease their shareholders.
The thing is though, regardless of its legal status in the US, the confederate flag does not really represent a noble cause. You can claim all you like that the confederacy did not fight for the continuation of slavery, but every historical document and source of information speaks otherwise. It simply is not true that the civil war was not fought primarily over slavery. It was a debate that had been raging for decades before the outbreak of the war, and in almost every single declaration of secession published from each state of the confederacy, upholding slavery was clearly stated as their primary cause. The south plainly and simply did not want to abolish slavery. It's understandable of course, considering the entire livelihood of the south dependent on slavery as an established institution, and I'm not entirely sure how anyone could deny this with the historical resources available to us on the internet today, but all the same, it's quite clear what the confederacy stood for, and it's quite clear to me that their cause was not one of honor or nobility.