Friday, October 11, 2013

SNAFU At Singtel

Situation Normal, All Fouled Up
The fire was so insignificant, it took only 20 minutes to put out. Or so we were told. Then the number of cables damaged was revised from 33 to 51 on Thursday morning, when normal service was supposed be restored by 6 am of same day. This was subsequently changed to 7 am on Friday for corporations affected, and 7 pm for residential services.

Something was already wrong when internet services went on the blink at around 3 pm on Wednesday afternoon. The Singtel hotline was no help, with their useless robotic responses and irksome elevator music. About 2 hours later, a pre-recorded message made reference to a fire at their Bukit Panjang exchange. Another couple of hours later, even the hotline gave up the ghost, inundated by irate callers. The radio or television broadcasts provided little or no clues. It was only much later at night that the scrolling news feed made cursory mention of the cable meltdown at Singtel. Onscreen the videos were about a fire in Bangladesh (?) or some disaster in other faraway countries. Could this be due to one clown's grouse about Singaporeans focusing attention on the roof collapse at a Jurong shopping mall, instead of the shoot out at a Nairobi shopping mall?

100 mobile base stations, 60,000 fixed broadband lines, around 30,000 mioTV customers, and around 30,000 voice lines were affected. As of Friday morning, the total number of cables damaged was updated to 149. As for the accounting of information coming in by dribs and drabs, we are told the fibre chamber - one of three at the exchange - measured "only 40m by 5m". Meaning it took a long while for someone to dare venture into the actual site to assess the exact scale of the disaster. Minister for Communications and Information Yaacob Ibrahim said he was “quite happy” with the way SingTel handled the disruption.

While waiting for the service LED on the modem to come back to life, many Singtel subscribers must have wished they have an alternative access to the world wide web.

When the data cables were laid for the distributed communication system (DCS) at Pulau Seraya Power Station, we were surprised that the industry standard dual pair of ethernet lines - one active, one on standby mode - was considered insufficient redundancy protection by the power engineers. They had insisted on another dual pair, routed separately, in case an errant forklift truck or excavator should cut into one pair, and trigger a national blackout. This was one wise career move.

The companies whose services were brought down by Singtel, including ATMs operated by DBS, UOB and OCBC, must have wished they had satellite dishes installed for internet access. Only corporations are permitted to have such equipment, private residential homes in Singapore are barred from receiving signals from the sky. Malaysia, Indonesia and Thailand have no such ban from this communication alternative.

28 comments:

  1. Wow, the damage is great. Are they trying to confuse us with misinformation? The Ethernet over twisted pair of cables is not a redundancy system but one that reduces attenuation over long distances.

    ReplyDelete
    Replies
    1. Reliability for TRACE MODE based SCADA/HMI PCs is assured by several network adapters support and by PLC communication lines redundancy. It assumes automatic switching-over of the server to standby network adapter in real time, in case of failure of the main adapter.

      Delete
    2. No. They are not trying to mislead you. They think that sinkies are daft enough to understand that these unfortunate event is either once in 50 years event or that they are deem unimportant compare to pappies' million dollar salaries. How can that be important when even news of ten of billions of loss by Temask Holding and GIC in financial fiasco can even be censored and downplayed by the pappies ?

      The only thing pappies want you to know is that opp parties fail to do ceiling cleaning even though we call them bluff.

      Delete
  2. CEO says sorry?

    ReplyDelete
  3. Snafu by Singtel.
    Settlement by Stat Board.
    Scandal (another one) by MFA.

    What else is not new under the sun these days?

    Me just waiting for those soldiers who keep spilling their stinking squawks to screw up some spunk and sue Mindef to court, just like those real Israelis soldiers who take to Supreme Court and demand IDF reforms for equitable conscriptions and treatment by their government and law makers. Bunch of sissy grumblers who prefer self-victimization than creating change for themselves.

    http://www.nytimes.com/2013/05/30/world/middleeast/proposal-for-ultra-orthodox-conscription-gains-traction-in-israel.html?_r=0

    ReplyDelete
  4. To lhl and his merrymen,

    please, if it's not too much of a trouble of your million-dollar time, could the minister produce a dossier of what happened.

    ReplyDelete
    Replies
    1. That's the beauty of it, since they may be under Govt control but are basically publicly listed, therefore how can million-dollar Minister be held responsible?

      Delete
  5. "This is a once in 50 years fire."
    "No amount of engineering can prevent this."
    "What to do? It's happened. Let's move on."

    “Please remember, when you lower cost than what is the average in the markets, that means you are incurring higher risk. So on that message I leave you to ponder about your next purchase."
    "Repent."

    ReplyDelete
  6. a clear lesson on why back-ups/alternatives/Plan B's are vital -- from cables to .... political parties who can govern.
    and this no matter how good u think what's being used is.

    ReplyDelete
  7. Can someone investigate why Lee Hong Yi who was on a PSC scholarship in MIT can straight away work for Google instead of serving his bond with the government? There could be more.

    ReplyDelete
    Replies
    1. The misdemeanor committed by HongYi pales in comparison to what the grandfather and father have done to Singaporeans. The best outcome is to ensure at least one spend his retirement permanently in Changi Resort and you can rest assure HongYi will make the effort to visit....

      Delete
    2. If President Tony Tan can arrange for his son to "serve national service" in a laboratory, what's there to stop Prime Minister Lee to arrange for his son to "serve" at Google?

      Delete
  8. Just imagine, instead of Singtel's internet exchange, what will happen if the house of the dominant political party is on fire? Never say never. As to everything, it is always safer to have back up,

    ReplyDelete
    Replies
    1. You mean the house is not yet on fire?
      I think it's already all burnt out.
      Just waiting for the daft 60% to wake up only.

      Delete
    2. Not just the daft 60% waking up, but to help fan the fire.

      Delete
    3. My champagne is waiting to be uncorked.
      Just waiting for the bucket to be kicked.

      Delete
  9. Move on, nothing to see, not serious enuf for old fart to flip in his deathbed and jump up, his brother's daughter in law is in charge there so everything must be a-ok. Even in bangladesh fires in garment factories are now reported as national event in the free press. Third world people deserve third world service and first world elites. Suck it up sheeple.

    ReplyDelete
    Replies
    1. brother's daughter-in-law?

      Delete
  10. A captive audience has very little choice in a monopolised society.

    ReplyDelete
    Replies
    1. Captive audience or stupid audience?
      Voting wisely to protect yourself also don't know how to do.
      Like this got hope or not?

      Delete
    2. In the event of an undesired outcome expected an iron fisted response. But do not worry we all know what the outcome's gonna be. You just cannot beat the SYSTEM...lah.

      Delete
  11. I think the current proposal of selling OpenNet to Singtel and handing control of the Next Gen Network infrastructure over to them should be evaluted carefully, in light of the company's handling of the current outage.

    Singapore has this tendency of putting her most prized eggs into the same basket, in the name of "efficiency" and "pragmatism". Time for a rethink perhaps.

    ReplyDelete
    Replies
    1. too lazy to try alternatives;
      too scared to lose control;
      too haughty to realise they're not the only ones who can do things.

      Delete
    2. What I really want to know:
      The profits of Temasek Holdings.
      What is the breakdown?
      a)Profit from investing in foreign shares
      b)Profit from buying assets from PAP gahmen and then re-selling at a profit later on.

      Delete
    3. The bottomless pot of gold is a large tax payer pool. Simple economy of scale. 6.9 is better than 5.9 and 7.0 is too many? A risk free enterprise.

      Delete
  12. The best part of this tragic-comedic fiasco is what the SingTel spokesman said when asked why there is no back-up, why it took so long to restore services and why there is no business contingency plan. His reply was that "this is a physical fire" and therefore hard to cope. WTF - is there any other type of fire? Unreal fire? Virtual reality fire? Play play fire?

    ReplyDelete
  13. The multi Billion $ losses incurred by Temasek and GIC in recent years can cover the cost of launching and maintaining one or more satellite in the sky for civilian and military purpose.
    NEH should rethink how he uses the 12B defence budget.

    ReplyDelete
  14. people shud not be so naive, first they say network is "resilient", then minister yakult said he was "satisfied". It is not likely there was no redundancy, far more likely some were directed to unca sam's global network of listening posts as interim workaround, banks, sheeple etc all are dispensable, big brother's "needs" are not negotiable. Same_ police police everywhere, except where they are needed to fight local crime. So what are they doing searching elsewhere? Simple- running errant for unca sam

    ReplyDelete