One Direction turn down X Factor judging offer
But Louis, 20, told The Sun: "We don't feel like we are in a position to be giving strong advice."
Simon, 52, had plans for the boys to star in an episode and conduct a master class with this year's hopefuls.
The guest slot would follow 1D's hugely popular appearances on the Australian and US versions of the show.
But the lads are probably also too busy to take time out to star in the show anyway.
Their new album, entitled Take Me Home, is due out on 12th November and they're currently promoting their latest single, Live While We're Young.
The video for the track broke records when it debuted on VEVO earlier this month.
It pulled in no less than 8.24million views - smashing Justin Bieber's record for his recent single, Boyfriend.
Steve Barnett, chairman of One Direction’s American record label, Columbia, said: “VEVO had more impact than anything else [for One Direction]. I’ve never seen anything work as quickly.”