Should I feel bad that my husband doesn't want a second baby?

I'm 24 years old, my husband is 25. We've been together 6 years and married for 4. In 2014 we bought our first home together. He's a carpenter in a union, and at the time I was a Customer service representative at a bank. Right after we bought our home, he regretted it. He became distant to me, and we had a lot of issues at that time. Then a couple months later I get pregnant. When I was a couple months pregnant I lost my job due to the bank downsizing, and my husband got laid off. So what use to be really good money from both of us, was now a low earnings. He wasn't laid off for long, but we was still having to pay bills with only one income. Last year, we decided that we needed to sell our home since money was tight. My parents has a fully finished basement, with bedroom, bath, kitchen and laundry room. So we decided to move in there while we sell our home. It's taken over a year. But we are finally closing and selling the home in almost a week. We are talking about moving a home onto our parents farm. But we won't start that for another year to save money. I brought up having another baby last night to my husband. Since our son is 2 and we're finally getting rid of our $900 mortgage payment now. But he said no. He didn't want to have our first baby and claims I forced him to. Even though when we was dating he told me he wanted a couple kids. When I asked him for a reason, he said he isn't having another baby while living here. And probably wouldn't have another for years. I don't want to wait till my son is old to have another one. I want him to have a sibling to grow up with, to have stuff in common. And I honestly didn't see the issue of having a baby while we're staying here. I really want another baby. My first pregnancy I pretty much did alone cause he wasn't very supportive. Is it wrong for me to feel this way? I was just really looking forward to selling our home cause I really thought we could try for another baby. I've been so crushed since last night.