The 81-year-old's time at the helm had coincided with the revival of the French men's national team as a force, with their victory at the 2018 World Cup followed by a run to last year's final in Qatar, which they lost on penalties to Argentina.
"Noel Le Graët announced... to the Executive Committee of the French Football Federation his decision to step down from his position," the FFF said in a statement.
"Philippe Diallo, vice-president, will act as interim president of the FFF until June 10, 2023, the date of the next federal assembly."
Le Graet's resignation comes 13 days after the publication of a damning report into management practices at the Paris-based FFF which had been commissioned by the French sports ministry.
"Considering his conduct towards women, his public comments and the governance failings of the FFF, Mr Le Graet no longer has the necessary legitimacy to run and represent French football," the report said.
Le Graet had already agreed in January to step down pending the outcome of the audit, which concluded that he should not return to the role because his "behavioural excesses are incompatible with the carrying out of his functions."
It was revealed last month that Le Graet, whose mandate was due to run until 2024, was being investigated for sexual and psychological harassment following allegations made by a female football agent, Sonia Souid.
He was already under pressure after making dismissive remarks in a radio interview in early January about France legend Zinedine Zidane's potential interest in coaching the national team.
French football has been in turmoil recently, with the French women's team coach, Corinne Diacre, also under fire.
Team captain Wendie Renard said last week she would not play at this year's Women's World Cup. According to a report in French multimedia outlet RMC Sport, Renard said she will not play for the national side as long as Diacre is in charge.
Fellow internationals Marie-Antoinette Katoto and Kadidiatou Diani also said they were taking a step back from the national team.