My bf divorced but keeps old "family unit"

Since the last year, I started to live with my bf who is 51 y.o, 15 years older than I, after 5 years distant relationship. I am new to the city and yet to make my own friends' network. He is divorced from his x half year back after 10 years of non-couple life, but parenting 3 kids together in the same house where he was coming back every weekend from other city where he is working. The smallest is 19 y.o. now, so they can start to live more on their own, then parents divorced and we started to live together in a new flat. My problem is that still children wish sometime to have "old family time" and even my bf enjoys to do that feeling "relaxing family time" with x-wife, while I am shut out, waiting alone in the house in a weekend. Even he is too busy with his work to have enough weekday/ weekend time with me. Initially reason to shut me out was that x-wife cannot see me psychologically. They made a divorce agreement that my bf can go once in two weekends to stay together with children, but both my bf and x-wife don't care if x-wife evacuates house or not when my bf visits the house. Children wishes to be with mother and not with me, as they feel "home" or "relaxed" more. He stayed overnights sometime spending time with x-wife and children which he stopped doing it after lots of fighting with me. He cannot clean out his room in his old house, he likes to spend sometime in the garden there. Legally he and x-wife share the house together and have to sell it after 5 years. Meanwhile if I have a contact with my x-bf in a friendship base, my bf gets mad and tells he cannot accept. He and x-wife keeps a shared account for the shared house and children, and he transfers money much more than it should be (as agreed by the divorce agreement), while he has no saving. X-wife earns much less than he, so he feels obligation to pay more and also by his character, he cannot say anything strong to x-wife. I feel he loves me and I also do, but I have big difficulties with him dealing with these issues. He tells things will go well by time because everybody has a good will and he doesn't feel necessity to tell anything important to his x-wife or children, he thinks everybody understands everything well. He tells I just have to understand the situation better and accept it. I feel children are accepting me well and individually I do not have problem and like to be good friends, accepting them well. But with his non guidance to them, they treated me cold to save their "old family time" while I did efforts to make new friendships with them, and now I got somehow traumatized with things happened so far. I wish he could make clearer line between his old family and show commitment for a family development with me, letting his children join, showing them the direction. It seems his priority is momental harmony, no conflict, no confrontation, and just does things which seems to be most peaceful. So he doesn't care what we have agreed or promised after lots of discussion. He knows it is not good to break the promises but he says that he did what was the best at each moment and he had a good will, so he was not wrong. He is getting older and it gives more difficult feeling to me, as I want to have my own children. How should I understand him, is there anything I can do to make things better, or should I move away ?