Can my employer give permission to my coworker to inspect my time cards and payroll sheets?

I have a colleague who is stalking me during work hours and keeping a diary of what I do, when I eat, who I speak with, etc. to try and get me fired. He has convinced my boss that I am guilty of transgressions that he himself is guilty of, such as leaving work early, not charging myself time, etc. My boss has given my colleague access to all my time sheets, time cards, personnel memos, etc. However, this worker has no one that reviews his time cards in comparison to his time sheet--he is "on his honor." While I have always worked required hours, it still bothers me that this person is given access to all my records for no other reason except to spy on me. Is this allowable?