Transport for London (TfL) turned down the plan, saying most taxi drivers did not support the initiative. But it has approved a taxi fare increase of 5.3% for this year.
The LTDA said the decision may lead to a shortage of black cabs for the Games.
Richard Massett, from the LTDA, said it had suggested to TfL the price rise, as a survey of its members had shown about 40% of drivers would take holiday during the Olympics.
"This was an idea for TfL to consult on, as a way to encourage drivers not to go away for the Games," he said.
"It has been tuned down, like we thought it would, so it is now up to TfL to make sure there are enough black cabs to cope."
Mayor of London Boris Johnson had previously described the proposal as a "major own goal" for taxi drivers.
And TfL said that when it consulted the taxi trade, the vast majority of responses from drivers were also against the increase as they believed it would reflect badly on taxi drivers and discourage custom.
As well as the 5.3% rise, TfL said it had also approved a 20p increase to the £2.20 "flag fall" - which is the minimum fare applied to the meter when a passenger gets into the taxi.
TfL's John Mason said: "The mayor and TfL are convinced that the world famous knowledge that taxi drivers worked so hard to earn will play a key role this summer in transporting Games spectators and visitors around the capital.
"This increase will allow them to continue to do so while providing value for money to passengers."