U.S. Cellular Service Blackout in Iowa, Illinois

[jones]

U.S. Cellular suffers service blackout in Iowa, Illinois
By Kelly Hill
Jan 17, 2006
CHICAGO-A power outage put a U.S. Cellular Corp. switching center near Cedar Rapids, Iowa, out of commission for several hours Monday. The outage caused disruptions in cellular service to the carrier's customers in eastern Iowa and one portion of western Illinois.
Cellular service went down around 6:45 a.m. CST and was restored by around 2 p.m., according to U.S. Cellular spokesperson Matthew Tiefenbrun. He said the cause of the power outage was not yet known and engineers were investigating what had happened.

http://www.rcrnews.com/news.cms?newsId=25348

[observer]

Originally Posted by jones

U.S. Cellular suffers service blackout in Iowa, Illinois
By Kelly Hill
Jan 17, 2006
CHICAGO-A power outage put a U.S. Cellular Corp. switching center near Cedar Rapids, Iowa, out of commission for several hours Monday. The outage caused disruptions in cellular service to the carrier's customers in eastern Iowa and one portion of western Illinois.
Cellular service went down around 6:45 a.m. CST and was restored by around 2 p.m., according to U.S. Cellular spokesperson Matthew Tiefenbrun. He said the cause of the power outage was not yet known and engineers were investigating what had happened.

http://www.rcrnews.com/news.cms?newsId=25348

Tuesday, January 17, 2006 12:06 PM CST
U.S. Cellular customers lose service
By RC BALABAN, Courier Staff Writer

WATERLOO --- A network outage caused area customers of U.S. Cellular to lose phone service for several hours Monday.

Jodi Valenta, Northeast Iowa sales manager for U.S. Cellular, said a power failure at the company's switching center in Cedar Rapids caused the network outage and the company was still investigating the cause of the power failure. She was not sure how many customers were affected.

Valenta said because most U.S. Cellular customers were in their home area, their phones would not have gone to roaming, or worked off another company's network. She said many customers came by her store asking about the problem.

"For the most part, they were very accommodating and understanding," she said.

The outage occurred early Monday, with the network being restored by 2:20 p.m.

U.S. Cellular is based in Chicago and is the nation's sixth largest wireless service carrier, with more than 5.2 million customers in 25 states.

[ndtoddman]

Hello Carriers...can you guys say "standby power"? If this was caused by a commercial power failure at the switch and they do not have standby power generators or at least battery back-up to handle an outage like this they have a poor network. The carriers must begin to harden their networks for commercial power failures. In my area, commercial power failures account for the greatest number of celluler network outages. Most sites have 4-6 hours of battery back-up, however few have engine driven generators to power the site for extended power outages. One note:Verizon has taken the steps to harden their networks. All new cell sites have standby generators installed and many existing sites are getting them. Thanks Verizon.

As for the statement about your handsets roaming onto another carrier durring your primary carriers failure. In theroy its correct but in reality its a bit misleading. My carrier: CellulrOne/Alltel had a software glitch during an upgrade at its Fargo, ND switch about a week ago. This upgrade was being done as part of the change over to Alltel. The glitch casued an outage to a large area for over an hour. Handsets roamed to Verizon however, you could not place calls. Verizon's network operations center must have seen the inrush of usage and blocked all Alltel phones to maintain thier system integrity for thier customers. No system is engineered to handle all of thier customers usage plus the total usage of another carrieres customers in the same area. Even thou my phone would access Verizons network it would not process a call. However, Verizon phone worked fine.

[royc]

Originally Posted by ndtoddman

My carrier: CellulrOne/Alltel had a software glitch during an upgrade at its Fargo, ND switch about a week ago. This upgrade was being done as part of the change over to Alltel. The glitch casued an outage to a large area for over an hour. Handsets roamed to Verizon however, you could not place calls. Verizon's network operations center must have seen the inrush of usage and blocked all Alltel phones to maintain thier system integrity for thier customers. No system is engineered to handle all of thier customers usage plus the total usage of another carrieres customers in the same area. Even thou my phone would access Verizons network it would not process a call. However, Verizon phone worked fine.

The reason you could not complete a call on Verizon is that the glitch at Alltel disabled Alltel's ability to authenticate your phone when Verizon's network checked to see if you were an authorized user. The same problem happened to some people (Verizon customers, if I remember correctly) who evacuated New Orleans to escape Hurricane Katrina. The loss of the network in New Orleans and its ability to authenticate customers from that area caused a loss of service for customers from the New Orleans area, no matter where they went. Their phones had a signal, but the network could not authenticate them. There were some threads about it on HoFo last September.

royc

[observer]

USCC announced yesterday that they would give 30 minutes to all customers involved in the blackout
woo-hoo!

[Dogma]

Originally Posted by observer

USCC announced yesterday that they would give 30 minutes to all customers involved in the blackout
woo-hoo!

So generous

[pbw]

Originally Posted by ndtoddman

Hello Carriers...can you guys say "standby power"? If this was caused by a commercial power failure at the switch and they do not have standby power generators or at least battery back-up to handle an outage like this they have a poor network. The carriers must begin to harden their networks for commercial power failures. In my area, commercial power failures account for the greatest number of celluler network outages. Most sites have 4-6 hours of battery back-up, however few have engine driven generators to power the site for extended power outages. One note:Verizon has taken the steps to harden their networks. All new cell sites have standby generators installed and many existing sites are getting them. Thanks Verizon.

As for the statement about your handsets roaming onto another carrier durring your primary carriers failure. In theroy its correct but in reality its a bit misleading. My carrier: CellulrOne/Alltel had a software glitch during an upgrade at its Fargo, ND switch about a week ago. This upgrade was being done as part of the change over to Alltel. The glitch casued an outage to a large area for over an hour. Handsets roamed to Verizon however, you could not place calls. Verizon's network operations center must have seen the inrush of usage and blocked all Alltel phones to maintain thier system integrity for thier customers. No system is engineered to handle all of thier customers usage plus the total usage of another carrieres customers in the same area. Even thou my phone would access Verizons network it would not process a call. However, Verizon phone worked fine.

Doubtful. Every MTSO, even rural ones based at cell site microwave hubs in the south have generators on site. What most likely happened is that the power went out, the generator did not kick over (I've seen this happen many times) and the batteries went dry and dropped the switch.

The real question is why isn't there a backup switch? Why was there not a second generator on site (N+1 redundancy)? I've seen this happen in the Norfolk/Newport News market. GTE/Verizon consolidated MTSOs (used to be two) into one. They then promptly botched a generic upgraded and lost service for several hours.

I mean even COX cable has two massive generators at their distribution hubs in Northern Virginia.

[Dogma]

Originally Posted by pbw

The real question is why isn't there a backup switch? Why was there not a second generator on site (N+1 redundancy)

I can't see a carrier owning a backup switch, but often Vendors (ie. Nortel, Lucent) have a mobile switch they can bring in during an emergency. An emergency usually constitutes the MTSO being totally disabled by weather, etc...

I agree there should be another backup generator, possibly a large mobile one that could be used for other MTSO's in the same circumstance.

[pbw]

Originally Posted by Dogma

I can't see a carrier owning a backup switch, but often Vendors (ie. Nortel, Lucent) have a mobile switch they can bring in during an emergency. An emergency usually constitutes the MTSO being totally disabled by weather, etc...

I agree there should be another backup generator, possibly a large mobile one that could be used for other MTSO's in the same circumstance.

Naa, not a backup MTSO as in a spare standby. What I meant was an MTSO, perhaps in the nearest native market that could take over for a downed MTSO. All it would require is an emergency routing change in LERG to have the circuits re-routed. It is not like the cell sites are directly wired to each MTSO.

[Dogma]

Originally Posted by pbw

All it would require is an emergency routing change in LERG to have the circuits re-routed. It is not like the cell sites are directly wired to each MTSO.

For this to be capable, everything would have to be setup exactly the same at each MTSO. Software, hardware and datafill would be another issue. Not to mention the "emergency swith" would have to have the additional capacity to take over another network. I could only see this being feasable for an outage that lasted more than 48 hours.

In my experience, cutting a few cell sites over to a different switch can be a major pain.