Okay, here is the answer.
These are the sets of three numbers that multiply to give 36, along with the house number (the sum of the three numbers).
1 2 18 add to 21
1 3 12 add to 16
1 4 9 add to 14
1 6 6 add to 13
2 2 9 add to 13
2 3 6 add to 11
3 3 4 add to 10
We don't know the door number, but the mathematician does. Let's say the door number was 21. The mathematician would have known that the ages were 1, 2 and 18. So if the door number was 21, the mathematician would have continued to the next house. But he didn't. He could not work out the ages. He had to go back for more information. The only reason he would do this is that the door number would not be enough. But if the door number is 21 or 16 or 14 or 11 or 10, this is enough. So it cannot be any of those numbers. The door number must be 13 (as there are two sets of numbers that add to 13). If the door number was 13, he would not know whether the ages were 1 6 6 or 2 2 9. And this is why he had to go back for more information. The knowledge that the youngest one is at Grandma's rules out 2 2 9, as there is no youngest one. So the answer is 1 6 6.
