(CNN) -- Military dive teams worked the dark, cold waters of the Yellow Sea on Wednesday night in a desperate effort to find nearly 300 people who remained missing after the ferry they were taking to a South Korean island resort sank with breathtaking speed.
Officials said four people were known dead and at least 164 had been rescued after the ferry Sewol swiftly listed and then capsized off the southwest coast of South Korea.
Nearly 300 others are missing, authorities said, but the exact number of passengers aboard the ferry was unclear amid the confusion of the rescue effort.
The bulk of those aboard were students and teachers from Seoul's Ansan Danwon High School heading to a four-day trip to Jeju, a resort island considered the Hawaii of Korea. More than 300 students and more than a dozen teachers from the school were on board, according to officials.
It was not immediately clear if the dead -- three males and a female -- were students, teachers, part of the crew or other passengers from the ferry.
The ship sank within two hours of its first distress call, which came just before 9 a.m., the semiofficial Yonhap News Agency reported. It's not known what caused the incident.
Helicopters, military vessels and fishing boats swarmed to the scene to help rescue passengers.
Survivors told CNN affiliate YTN that announcements on board the ferry, which also carries cars and shipping containers, ordered passengers to remain where they were after the ship began to violently list.
Others told passengers to jump into the water. Rescued student Lim Hyung Min was among those told to jump.
He told CNN affiliate YTN that he heard a loud bump just before the ship began to list and several off his classmates were flung off their feet. The crew ordered them to don life jackets and jump into the ocean, he said.
"I had to swim a bit to get to the boat to be rescued," he said. "The water was so cold and I wanted to live."
Water temperatures in the area are between 50 and 55 degrees Fahrenheit (about 10 to 13 degrees Celsius), CNN meteorologist Brandon Miller said.
Passenger Kim Seung Mok said that despite his efforts and those of others, he couldn't get to several passengers on one of the decks.
"I stayed till the last to rescue people at the hall," Kim told YTN. "But the water was coming in so fast (that) some didn't make it out."
Water temperatures, swift currents and low visibility appeared to be complicating the massive rescue operation, which involved dozens of South Korean military divers, sailors, marines and police officers.
The U.S. Navy ship USS Bonhomme Richard, on routine patrol in the area, dispatched its helicopters to aid in the rescue and was headed to the scene, the U.S. Navy said.
Divers from the South Korean navy searched three of the ship's compartments but found no survivors or bodies, Yonhap reported. Another dive team was expected to continue the search shortly, the news agency said.
At Ansan Danwon High School, parents clutched their cell phones in an agonizing wait for a call from their children. Officials posted a list of names. Once a confirmation of a rescue came, they circled that name.
At one point, the school announced that all students had been rescued but soon backtracked, to the parents' wrath.
What could have caused the ship to sink so rapidly remained a mystery. The weather was clear at the time of the accident.
Peter Boynton, a retired U.S. Coast Guard captain, said the speed with which the ship sank suggested it had sustained "major damage." He also said that if the ferry's car deck had been breached, it could have quickly swamped the ship.
Battling against darkness, cold, swift currents and trying to find their way through a damaged, upturned ship, rescuers are "up against every sort of obstacle," said David Gallo, director of special projects at the Woods Hole Oceanographic Institution.
"It's just an absolutely, positively horrific situation," he said. "It's nightmarish."
