My first thought was the two part story about the lighthouse that the family temporarily moves into, where they encounter a ghost (and a ghost ship).
These are episodes 25 and 26. The ghost is similar in appearance, but it isn't the one.
Then I thought this might be a scene from the first house moominpappa built in his youth. The vibe in that house was a lot like the lighthouse.
This was correct. The scene is from the third episode about Moominpappas youth, or episode 68.
The scene occurs at 9:30. Pappa and the gang aren't scared of the ghost, so he essentially became Pappas roommate. The ghost takes up knitting, since he can't seem to scare anyone.