Can Department Head take adverse action for a verbal agreement with a immediate supervisor?

I'm a Federal employee working for the US Navy. Department Head is threatening to take disciplinary actions against me for leaving the office early claiming that I was defrauding the government. I explained to him that there was a verbal agreement between my immediate supervisor and me but he would not take that into consideration. My verbal agreement with my immediate supervisor was that I would start early and finish early at the end the day but still putting in the 8 hours of work. He (the department head) stated that my starting time was not actually changed in our time management system (SLDCADA), therefore, it was illegal for me to go home early. Knowing that my current work time hours was verbally approved by my immediate supervisor, does he really have any grounds for taking disciplinary actions against me for this? Also, do I need to talk to a lawyer to help me defend my case ( which now he is turning into a misconduct after conducting his preliminary interview with me)
official working hours 0630 AM to 300 PM.
Due to bad traffic, I entered into a verbal agreement with immediate supervisor (start at 5 AM and leave 130 PM).