To escape a body of mass you need to have enogh velocity (kinetic energy) to overcome the gravitational pull of that body. You can imagine it like a ball sitting in a bowl. With little velocity it will just roll back and forth but if it's fast enough it can roll out of the bowl and escape it's influence.

That critical speed is called "escape velocity" and it depends on mass and distance from a body. The escape velocity of earth (from the surface) is about 11.2 km/s and the sun's escape velocity (from earth orbit) is about 42.1 km/s. Earth orbits around the sun at about 29.8 km/s. If you launch in the direction of Earth's orbit, you will orbit the sun already at about 41 km/s, so you "only" need 1.1 km/s more to escape the sun, too.

If you tried to reach the sun, you could launch in the opposite direction leaving you orbiting the sun at about 18.6 km/s. Since there is almost nothing in space you won't slow down from friction and the orbit won't decay. Instead you'd have to accelerate opposite the direction you're traveling. Now, calculating exactly how much you'd need to decelerate isn't trivial since you don't want a stable orbit but an elliptical orbit that just touches the sun at the closest point (perihel). I don't know how much deceleration that takes, but it's propable that it's easier than accelerating by 1.1 km/s to escape the sun.