My partner won't tell his ex wife about me and our baby

i have been with my partner for 2 years , we have both been previously married and both have children from our previous marriages. we have moved in together last year and now also have beautiful 5 month old baby girl...... we should be very happy yes... well i am all apart from one thing... he has been divorced for 3 years and has 2 young children who he visits every weekend( they live 100 miles from us), he loves these children very much and i think it is great he still sees them every week . but... his ex wife know nothing about me or our new baby i didnt mind in the beginning as i understood that he needed to be sure our relationship was strong before he told them , but we have been together 2 yrs have bought a house together and now a 5 month old baby together ........... he keeps saying the children are too young to understand ( they are 5 & 3 ) but when is the right time??? i have met all of his family and they are lovely and we get on great , oh yes he doesnt let his family meet his children since his divorce either ( they think she wont let them visit but he has told me its his decision).. i need to know what to do , every sunday he goes there at 8 in the morning spends the whole day there hen comes home about 5 in evening during this day i am not allowed to call him , he calls his children every day on the phone too but always goes into the other room to do it . am i being unreasonable by expecting him to tell them about us ?? feeling more and more angry about it as the weeks go on

you sound like a good person, you also sound like you need to hear what you already know. What he is doing is in nones best interest he is lying to his kids you and everyone. he is having a relationship in some way w his ex still. he is robbing you and his children of family time together. Im sorry but i can assure you if all you say is true in then he isnt being honest.