O'Heir was born in Chicago, and is a graduate of the Thornton Fractional South High School in Lansing, Illinois, and of Loyola University Chicago. O'Heir was active in the Chicago theater during the late 1980s and early 1990s, training and performing improvisational comedy at Chicago's Second City and as a member of the six-person sketch comedy troupe "White Noise", which formed in 1987. The group wrote and produced comedic plays that often employed bizarre humor or black comedy. Starting in November 1988, O'Heir appeared in White Noise's production of The Book of Blanche, about a woman who falls through a television screen and ends up in a fantasy world influenced by various television show genres. In July 1989, O'Heir appeared in the Chicago premiere of the Tom Griffin play The Boys Next Door at the Edgewater Theatre Center. O'Heir played Norman Bulansky, a childlike middle-aged mentally handicapped man who works at a doughnut shop.
Starting in 1990, O'Heir appeared in White Noise's Stumpy's Gang, a one-act black comedy play by Patrick Cannon. The show played at Stage Left and Strawdog Theatres in Chicago, where it developed a cult following. O'Heir played Frank Bubman, the janitor for a genetics laboratory whose job is to destroy the unsuccessful experiments, which are portrayed by puppets. Frank secretly uses the experiments to stage private shows reminiscent of early television programs. Mary Shen Barnidge of the Chicago Reader said of his performance, "The grotesquely sexy Jim O'Heir adds a new dimension to the role of the enfant terrible with his uninhibited and enthusiastic portrayal of Frank, who will break your heart even as he turns your stomach." Ernest Tucker of the Chicago Sun-Times described O'Heir's performance as very funny and "unforgettable", and said he "held together this funny yet sick apocalyptic fantasy". Stumpy's Gang closed in the fall of 1991. Also with White Noise, O'Heir appeared in the play Ad-Nauseam, about a pair of writers who create an ad campaign about a character named Rim Shot, played by O'Heir. Rim Shot, dressed as a gladiator costume made up of bristle brushes and toilet detritus.
O'Heir has appeared in several television shows throughout the 1990s-2000s, including 3rd Rock from the Sun, Malcolm in the Middle, Just Shoot Me, Star Trek: Voyager, Suite Life of Zack and Cody and ER. O'Heir appeared in the 1996 baseball comedy film Ed and 1998 fantasy television film Harvey. Starting in 2000, O'Heir starred in a regular role in the Comedy Central series Strip Mall as Harvey Krudup, the unattractive husband of protagonist Tammi Tyler, who was played by Julie Brown. The character was the owner of the Starbrite Cleaners business, and Tammi married him under the false assumption that he was rich. After learning Harvey is not wealthy, Tammi makes numerous attempts to have him murdered.
In November 2003, he made a guest appearance in the tenth season episode of Friends, "The One With the Birth Mother", where he played the manager of an adoption agency. On November 28, 2004, he appeared in the Boston Legal episode "Loose Lips", where he played a man fired from being a department store Santa Claus because he is a cross dresser O'Heir also continued appearing in plays during this time, and played the inmate Dale Harding in Dale Wasserman's stage adaption of One Flew Over the Cuckoo's Nest at The Theatre District in Hollywood, California. Backstage writer Terri Roberts said O'Heir stood out among the show's cast. He also appeared in the 2007 family drama film Welcome to Paradise.
Starting in 2009, Jim O'Heir became a regular supporting cast member of the NBC comedy series Parks and Recreation. He originally auditioned for the role of Ron Swanson, before being cast as Jerry Gergich, a clumsy employee at a local government parks department who is often routinely mocked by his fellow co-workers. Although the character has been featured in the series since the first episode, he started to become more fully developed and play larger roles in the episodes during the second season. The episode "Park Safety", in which Jerry pretends to have been mugged in order to avoid being made fun of for an injury, was focused almost entirely around Jerry and drew positive reviews for O'Heir's comedic performance.
In 2011, O'Heir made a guest appearance on the NBC comedy-drama series Parenthood as a bar owner. In 2013, he guest-starred on Good Luck Charlie as a couch surgeon and the owner of a couch store, and later in the year he had a main role in the Ion Television movie My Santa, as Jack, Santa's right-hand man. That same year, he appeared on Jimmy Kimmel Live as real-life Toronto mayor Rob Ford, who had recently been involved in a crack-smoking scandal. O'Heir will be playing Scientist Hathaway in the upcoming movie, Range 15. In 2016, he voiced the character "Aiden" on Harvey Beaks, the grandfather of the titular character. In 2016, he appeared as Sheriff Reynolds in two episodes of Brooklyn Nine-Nine and as Constable Jim in four episodes of the YouTube series Edgar Allan Poe's Murder Mystery Dinner Party.