How can I be a "temporary worker" after nearly 4 years at the same job, same place?

I worked for a security company that has contracts for most of the security for Shaw Industries. Shaw has been replacing their own security guards (who are fully benefited, well compensated employees) with guards from my former company who are paid minimum wage with no benefits to do the exact same job. I worked there from August '08 to February '12, working well over 40hrs each week.
When my job site was shut down without notice, they had no work left close enough to me so I had to file for unemployment benefits. My benefits were denied based on the fact that I was considered a "temporary employee" and failed to contact my employer after the "completion" of my previous assignment.
I have appealed the decision, but I am worried that although this practice is obviously wrong, I know full well that right/wrong has nothing to do with the law. Can they do this?