More than 4,000 people have crossed from France so far this year, Home Office figures show.