AceShowbiz -

Thomas Gibson shared his side of story after he was fired from "Criminal Minds" for allegedly kicking co-writer and producer Virgil Williams during a heated argument. "It seems like they're trying to erase me from the show. That hurts," he told PEOPLE.

"We were shooting a scene late one night when I went to Virgil and told him there was a line that I thought contradicted an earlier line," the actor recalled the on-set altercation. "He said, 'Sorry, it's necessary, and I absolutely have to have it.' "

A frustrated Gibson then told the other actors about what happened. "He [Williams] came into that room and started coming towards me. As he brushed past me, my foot came up and tapped him on the leg," Gibson said. "If I hadn't moved, he would have run into me. We had some choice words, for which I apologized the next day, and that was it. It was over. We shot the scene, I went home - and I never got to go back."

"I feel like it took years to make a good reputation and a minute to damage it," the 54-year-old star said. "My pride and reputation are hurt, but in the end I know the good work is what people will remember. I just need more opportunities to do good work and be a good guy."

"I'm using this time to be with my kids, and I look forward to what's next," the father of three added. "Maybe I'll do a comedy, or a play, or direct, or produce."