The win boosted the Colts to 4-3, and gave Luck one more win than his predecessor, Peyton Manning, chalked up in his entire rookie season with Indianapolis.
"Andrew, I thought, was fantastic," Colts interim coach Bruce Arians said.
The Colts now have beaten Tennessee in seven of eight overall and seven of the past 10 in Nashville.
Luck led the Colts on a pair 80-yard touchdown drives. The No. 1 overall draft pick set up Delone Carter's 1-yard TD run that tied it at 13 with 3:24 left in the fourth quarter.
In overtime, Ballard capped the drive, taking a screen pass and going up the left side where the rookie got his feet knocked out from underneath him as the Titans tried to get him out of bounds. Ballard crossed the goal line upside down with his head hitting the pylon for the TD. Officials reviewed the play before ruling it a score.
"They were all around, and I just knew I had to get in," Ballard said. "The only way I could get in from the 5 was to jump so I did."
Luck worried the ball might be intercepted.
"I threw it and I saw a Titans uniform flash and I thought, `Oh, God, this may be the end of the game in the wrong way," Luck said. "What a wonderful individual effort by Vick Ballard to get it in there."
The loss left the Titans (3-5) stunned and shocked after blowing a chance at their third straight win.
"I don't really understand how we lost the game," Titans quarterback Matt Hasselbeck said.
After Carter's TD run, Tennessee moved down the field and Hasselbeck overthrew tight end Jared Cook on what would have been a touchdown. Then Hasselbeck threw incomplete before coach Mike Munchak chose to punt with 56 seconds left rather than let Rob Bironas, who beat the Colts with a 60-yarder on this field in 2006, try a 58-yard field goal with a stiff wind at his back.
Bironas, who kicked two field goals earlier and missed a 45-yarder into the wind, said only that it was a coaching decision.
Tennessee thought it had recovered a fumble that would allow Bironas to win the game in regulation. Colts fullback Dwayne Allen was stripped of the ball after a catch, and Titans cornerback Alterraun Verner recovered at the Colts 28. Officials ruled Allen down due, and the Colts kneeled down to run out the final seconds to reach overtime.
"It's a huge play and huge time of the game," Munchak said. "You wish it had gone the other way where you let the play go, and they can decide if the guy is down by contact. When you rule forward progress, then you can't review the play."
The Colts won the toss, took the ball and never gave it back.
Donald Brown, who had eight carries for 41 yards in regulation, got the ball on the first six plays of overtime and ran for 39 yards. Facing third-and-8 from the Titans 36, Luck stood in the pocket and found Reggie Wayne for a 20-yard completion. A play later, Luck tossed the screen, and Ballard did the rest to finish off the victory.
Indianapolis outgained Tennessee 457-339, and Luck was 26 of 38 for 297 yards. Adam Vinatieri also kicked two field goals, but Arians had plenty of praise for the rookie quarterback.
"He is a big strong young man, and he can throw the football from any position accurately," Arians said. "He did a good job of avoiding rushes and making completions, especially on third down."
The Titans led most of the game, tied only at 3-3 in the first quarter and late in regulation even without left tackle Michael Roos due to an appendectomy. Right tackle David Stewart and right guard Leroy Harris each hurt right knees on the opening drive. Stewart returned, though Harris never did.
Hasselbeck was 22 of 29 for 236 yards with a TD pass that was the 200th of his career making him the 32nd player in NFL history to reach that mark. Chris Johnson ran for 99 yards.
But the Titans came in giving up more points than any team in the NFL, and they just couldn't make the stop to give their offense one last chance. They sacked Luck twice. Michael Griffin got an interception and also blocked a 37-yard field goal at the end of the first half.
It wasn't enough.
The Titans pressured Luck repeatedly in their first shot at their division rival's new quarterback. Luck had a couple passes batted down but found open receivers time and again.
Colts CB Vontae Davis sprained his left knee on the opening drive and did not return. He was replaced by Cassius Vaughn. ... Colts TE Coby Fleenor hurt a shoulder in the first half and had one pass thrown at him in the second half. ... Bironas now is the franchise career leader with 201 field goals. Al Del Greco had 200 field goals for the Oilers-Titans. ... The Titans last blocked a field goal Dec. 5, 2010, when Jacob Ford had a block against Jacksonville.