The original county seat was located at Mannfield, and later moved to Inverness, which was named by a settler of Scottish descent. It appears that Mannfield was intended as a temporary county seat only. The county commissioners originally met at a local church, while court functions were conducted in the Moffatt and Gaffney residences, the latter rented for the sum of $19 per month. Judge E. C. May moved to Citrus County in 1892. At this time the county seat had just been moved to Inverness in a referendum that Mannfield proponents questioned. May writes that an injunction was sought from the closest judge, sitting in Dade City, but he had taken the train to Tampa by the time the rider arrived from Citrus County. Eventually a writ was obtained but the rider fell off his mule and could not serve the papers until after the Mannfield courthouse had been stripped of its records. Judge May also describes the "new wood courthouse" in Inverness with "the wire grass ... still living under it," and situated on a block otherwise covered with freshly-cut pine stumps.

     Both views of the original Inverness courthouse described by Judge May are from Hampton Dunn's 1976 history of Citrus County, Back Home. In one, county prisoners wearing striped uniforms are depicted maintaining the town's Main Street. In 1911 the wooden courthouse was replaced, on the same spot, by a stone structure. The architects, J. R. MacEachron and W. R. Biggers, reportedly used the Polk County Courthouse as a model; the style has been described as "eclectic, with elements of Neo-Classical, Italian Renaissance, Prairie School and Mission styles." The building includes a copper cupola with a clock face on each of the four sides, topped with a belvedere with miniature columns. Construction was by the Read-Parker Construction Company, at a cost of $49,965, plus an additional $875 to move the old courthouse. The black-and-white postcard view dates from the early 1960's. The building still stands in downtown Inverness, although it is no longer used for judicial functions.

The yellow-brick 1912 Citrus County Courthouse, which replaced the original 1892 wood structure, recently underwent restoration under supervision of the Citrus County Historical Society.  After peeling off layer upon layer added over the years, workers uncovered terrazzo floors and marble wainscoting.  Transom windows that had been painted over and nailed shut were refinished and re-hinged.  Old photographs of the courthouse gave clues to what the hidden walls, floors, and ceilings looked like.  Society members and architects even watched old reels of the 1961 Elvis Presley movie Follow That Dream, in which the closing courthouse scene took place in the second floor courtroom.