Tag Archives: big data

Is the cloud the right spot for your big data?

Is the cloud a good spot for big data?

That’s a controversial question, and the answer changes depending on who you ask.

Last week I attended the HP Big Data Conference in Boston and both an HP customer and an executive told me that big data isn’t a good fit for the public cloud.

CB Bohn is a senior database engineer at Etsy, and a user of HP’s Vertica database. The online marketplace uses the public cloud for some workloads, but its primary functions are run out of a co-location center, Bohn said. It doesn’t make sense for the company to lift and shift its Postgres, Vertica SQL and Hadoop workloads into the public cloud, he said. It would be a massive undertaking for the company to port all the data associated with those programs into the cloud. Then, once its transferred to the cloud, the company would have to pay ongoing costs to store it there. Meanwhile, the company has a co-lo facility already set up and expertise in house to manage the infrastructure required to run those programs. The cloud just isn’t a good fit for Etsy’s big data, Bohn says.

Chris Selland, VP of Business Development at HP’s Big Data software division, says most of the company’s customers aren’t using the cloud in a substantial way with big data. Perhaps that’s because HP’s big data cloud, named Helion, isn’t quite as mature as say Amazon Web Services or Microsoft Azure. But still, Selland said there are both technical challenges (like data portability, and data latency) along with non-technical reasons, such as company executives being more comfortable with the data not being the cloud.

Bohn isn’t totally against the cloud though. For quick, large processing jobs the cloud is great. “Spikey” workloads that need fast access to large amounts of compute resources are ideal for the cloud. But, if an organization has a constant need for compute and storage resources, it can be more efficient to buy commodity hardware and run it yourself.

Public cloud vendors like Amazon Web Services make the opposite argument. I asked Amazon.com CTO Werner Vogels about private clouds recently and he argued that businesses should not waste time on building out data center infrastructure when AWS can supply it to them. Bohn argues that it’s cheaper to just buy the equipment than to rent it over the long-term.

As the public cloud has matured, it’s clear there’s still a debate about what workloads the cloud is good for and which it’s not.

The real answer to this question is that it depends on the business. For startup companies who were born in the cloud and have all their data in the cloud, it will make sense to do your data processing in the cloud. For companies that have big data center footprints, or co-location infrastructure set up, then there may not be a reason to lift and shift to the cloud. Each business will have its own specific use cases, some of which may be good for the cloud, and others which may not be.

 

MCTS Training, MCITP Trainnig

Best Microsoft MCTS Certification, Microsoft MCITP Training at certkingdom.com

The downside to mass data storage in the cloud

The ability to access Dropcam video footage in the cloud is indicative of a broader trend in cloud computing that is eating away at privacy.

The cloud can be an enormously cost-effective way to increase storage and computing musculature, and also, sadly, a way to further add misery to those seeking privacy—or who just want to be left alone. It’s rare to see organizations stand up and shout, “we’ll not give your data to anyone!” or “the life of all stored data, except opt-in assets you want us to store, is always 90 days!” or “yes, we can determine in absolute certainty that your data has been erased to protect you and your identity.”

The cloud, in some warrens, has become a storage ground for the various factories of “big data,” whose ideals are generally to sell things to consumers and businesses. Correlating facts is huge. Ask Target, whose insight into discovering pregnancies helped them capture a nicely profitable market in the pregnancy and new mother world. Smart, you say. There is a downside to this.

Striking while the iron is hot is a great idea. This means harvesting information on searches to be correlated into ads at the next site you visit. Facebook and Amazon are famous for this, and it’s a huge amount of Google’s total business model. Google’s purchase of Nest last year, which gleefully rats out your utility use patterns, also meant the acquisition of Dropcam.

As ace reporter Sharon Fisher reported at TechTarget, Dropcam’s users allow cameras to send their data into Dropcam’s cloud, where it is archived seemingly indefinitely, to the delights of users, police warrants, and security monitoring individuals, who see the surveillance results at will, from any reasonable IP address. It’s inferred that some users monitor Airbnb suites (shouldn’t they disclose this?) and apparently users forget there’s a camera on and do, well, silly things that they may not want captured on digital film.

Google’s storing this sort of info, Amazon will be listening with Echo, and who knows what Siri knows but isn’t saying. This amounts to a comparative heap of very personal information, as though these were robots whose knowledge base was contained inside the physical unit we see on-premises, but it’s not—it’s in the cloud and not only hack-able, but perhaps being used to analyze us, sell us something, or maybe worse, refuse to sell us something or to used against us in a court of law.

Is this data tagged so someone knows to kill it? Is there a metadata tag saying this file or this datablock expires on April 19, 2017? Often it’s tied to an account. Does this data get reused somehow? Video, audio conversations scrubbed for keywords? Much is up to the user agreement, and what happens if you’re, say, a medical provider that’s amassing large quantities of personal medical data? Can that be used? Yes, an attorney would say, “stop right here, and let’s disambiguate these questions.” Clear as mud.

The average civilian has no “bill of rights” that’s common to these online personal information services, whose data is accumulated in cloudy locations. Murky might be a better way to think about it. You want to trust data storage providers – one wants to believe that data sources are somehow bulletproof – but with huge, emblematic recent breaches of retailers, insurance providers, and university alumni databases, that’s not so easy. In reality, some have already been hacked and we just haven’t discovered it yet because no one’s offering the information on dark markets….at least right now.

Is there a way for the app industries to have a common agreement about what can be shared, what is a reasonable life expectancy for personal data, how and to what extent personal data can be actually anonymized, and how data destruction can be audited to even a private detective’s satisfaction? I wish there were answers.


 

 

MCTS Training, MCITP Trainnig

Best Microsoft MCTS Certification, Microsoft MCITP Training at certkingdom.com

How to make the most of an IT buyer’s market

After years of fighting tooth and nail with vendors for meager price discounts or modest service-level agreements, IT has seen the tables start to turn: Sweeping changes are reshaping the vendor landscape, shifting negotiating power from stingy service providers to savvy CIOs.

Practical advice for you to take full advantage of the benefits of APM and keep your IT environment

At the center of this sea change are trends such as cloud computing, social media, data analytics, remote monitoring, automation and mobility. Whether it’s manufacturers opening sensor-operated plants or healthcare providers using remote patient-monitoring systems, organizations are acting fast to seize new opportunities and satisfy customer demands. And as the need for agility increases, cloud-based computing is booming: Infrastructure as a service and business process as a service are the two fastest-growing segments of the IT services market, expanding 44.9% and 12.4%, respectively, in 2014, according to Gartner.

Just ask William Graff, senior vice president at Cerner Technology Services. “We’re spending a lot of time with specific internal business units looking at cloud solutions that traditionally we would have hosted on our own data center. But because of business pressures to move rapidly, we’ve selected a handful of cloud providers over the last year,” he says.

Decisions like that at companies of all kinds are adding an infusion of new, agile cloud providers into the average company’s mix of more traditional vendors.

Combined with the need for speed is an increased awareness of high-tech products and services. “The buyer’s paradigm has changed dramatically over the last several years,” says Keith Lubner, CEO and a managing partner at Channel Consulting, a Philadelphia-based management consulting firm specializing in vendor relations. “Access to information and content is so dramatic that buyers are more astute than ever before. They have instant access to information on every single product out there — they’ve flipped the whole sales cycle on its head.”

Armed with information and eager to take advantage of fast-acting cloud, analytics and mobile technologies, CIOs are voting with their wallets: A staggering two-thirds of respondents to a recent Gartner CIO survey said they expect to change primary suppliers by 2017.

Desperate to stay on top, traditional IT vendors are responding by tossing out their typical IT sales models to offer flexible subscription services, shorter sales cycles, unprecedented product innovation and personalized service. And that’s creating a once-in-a-career opportunity for savvy CIOs: a chance to negotiate huge price cuts, packaged deals, favorable contracts and unique partnerships with big-time vendors once too busy to return calls.

Tough Customers
Jim Forbes is a perfect example of the type of technology executive that’s keeping vendors up at night. The CTO at University Health Network (UHN) in Toronto, Forbes had for years relied on standard criteria such as “functional requirements and server compatibility” to select technology solutions. But a desire for a more scalable cloud-based tool recently convinced him that it was time to switch IT management vendors.

“Our new thinking sent us in a very different direction,” says Forbes. “We ended up going with an entirely different vendor — one who really wasn’t on our radar. There was some nervousness, but we recognized that this was the right strategic move, and looking back, it was a very good choice.”

James Cole, CIO at First National Bank of Omaha, also has a set of purchasing priorities that could cause vendors to panic. Once susceptible to industry buzz and product hype, Cole says he now finds himself taking a more business-oriented approach to vendor selection by asking, “Where does a solution fit into my organization?”

Today, IT staffers meet with the retail bank’s business-line leaders five days a week “to understand what their needs are and bring IT solutions to them,” Cole says. “We’re becoming more in tune with our core business, helping our business-line leaders with their needs and then going out into the market and determining who best can solve that need.”

But shifting the focus from acquiring tech tools to discovering business solutions is also changing the nature of the CIO-vendor relationship — to the CIO’s advantage. Increasingly, vendors are being asked to be a partner rather than simply a provider.

Cole points to the First National Bank of Omaha’s four-year relationship with Client Resources Inc. (CRI), an IT talent and solutions provider. Once considered a supplier of temporary labor, the midsize vendor has evolved into a key collaborator with the bank’s IT department, Cole says. For example, CRI recently worked hand in hand with the bank to design and develop a mobile app. “It became this great partnership,” says Cole. “If you were in a room with us, you’d have a hard time knowing who was the First National employee and who was with CRI.”

Even tech vendor titans are shedding their hands-off reputation for a hand-holding approach that they hope will help them retain customers. Seven years ago, Cole says, the bank’s dealings with Oracle could have been described as “a catalog relationship” involving occasional database orders. Today, a senior executive from Oracle is dedicated to helping First National Bank of Omaha with some seemingly minor IT projects, such as developing a better login process for mobile employees. “It’s very much a collaboration,” says Cole. “Oracle is looking at their business model differently now and seeing customers as a relationship rather than a product sale. It’s just interesting that Oracle is listening to us.”

Power Shift
In fact, whereas power-wielding IT vendors once shaped CIOs’ buying behavior, CIOs are now having a profound impact on the way a vendor approaches everything from customer service to product development. Says Graff: “When we enter into a long-term agreement with a vendor, we expect that our voice as an end-user community will be heard and that we’ll influence changes and enhancements to a product.”

Cindy Waxer
But as CIOs and vendors increasingly become bedfellows, the IT world is drafting its own version of a prenup. For instance, when UHN began vetting vendors for a new managed service contract, Forbes insisted that each interested party develop a five-year plan illustrating how unit costs might change over time. Vendors that promised cost-per-unit decreases earned extra points.

Another way Forbes gained an upper hand in negotiations was paying research firms, including Gartner and PricewaterhouseCoopers, for market analysis on IT service prices such as help desk costs and server fees — industry benchmarks that provided “a viable opportunity to negotiate cost reductions right upfront before signing a contract,” he says. “We’re doing more strategic thinking as part of the RFP process as opposed to just writing an RFP and throwing it out onto the street.”

Such an informed approach to negotiating pays dividends, according to Cole, who says he once talked an IT vendor into reducing the price of a system by $3 million. “We build performance milestones into all of our contracts,” he says. “We also do a very good job of negotiating [financial] holdbacks so that we don’t feel like we’re paying for a service well before it’s delivered.”

Faced with dwindling bargaining power and better-educated customers, many vendors are sweetening the pot by offering cost-effective bundles of services. For example, a vendor specializing in email encryption technology might try to package its tool with an Exchange server and high-margin services such as consulting on compliance issues.

In fact, Lubner says peddling packages is “the only way for vendors to differentiate themselves and provide more value to the buyer” these days.

Forbes agrees. Just as the federal government has been slowly embracing a shared services strategy, he says, the healthcare industry is inching toward a similar model, where multiple hospitals, clinics and laboratories will agree to share the funding and resourcing of key IT services to cut costs and improve efficiency. “There’s a lot of opportunity to save money and reinvest the subsequent savings back into healthcare if we only shared some of these IT services,” says Forbes, adding that the vendors that are most likely to come out ahead are those that “recognize the market is shifting and respond by packaging their software.”

Sharing the Legal Load
But greater collaboration, bundled IT services and high performance standards aren’t the only changes in the IT landscape helping to create a buyer’s market. Organizations are demanding that even legal issues be treated as shared responsibilities rather than matters to be hashed out by bloated legal departments. After all, says Cole, “if you just have two sets of legal teams talking, you’ll reach an impasse very quickly.”

Cole should know. In the first four months of this year alone, he’s had to tackle questions of intellectual property with at least three different vendors. That’s because, in these litigious times, it’s becoming increasingly common for unwitting end users to wind up entangled in patent infringement suits. For example, a hotel chain that offers its guests free Wi-Fi might find itself involved in a patent suit simply because it relies on server technology that has come under legal fire.

However, whereas in the past vendors would simply scoff at the notion of shared liability, Cole says there’s more willingness now to address issues such as intellectual property as a mutual business challenge rather than as a legal technicality.

“We need to have protection so now it’s a negotiation to determine how much liability a vendor is willing to give up and how much risk am I willing to accept,” says Cole. “It’s become a business discussion, not a legal discussion. In fact, it’s a very common conversation in the IT community today.”

That’s not to suggest, however, that vendors are simply rolling over and letting customers call all the shots. For example, Graff points out that Kansas City, Mo.-based Cerner is both a buyer and a provider of IT services and says that, no matter how profitable a project might seem, the company will “never sign a deal [as a provider] where we can’t deliver on what we’ve written into the contract.”

But that’s not all. Cole says most organizations do recognize and respect a vendor’s need to turn a profit. “Recently we negotiated a deal where it came down to both sides saying, ‘Here’s what I’m willing to give you in terms of profit and here’s where I need to be in terms of expenses,'” he recalls. “That’s a partnership where we shook hands. But had we dug in our heels and said, ‘Here’s all I’m going to give you,’ we both would have left with a bad taste in our mouth.”

After all, there’s no telling when the pendulum will swing back to a seller’s market. All the more reason, says Graff, for savvy CIOs to make sure “it’s a win-win situation for both parties.”


 

MCTS Training, MCITP Trainnig

Best Microsoft MCTS Certification, Microsoft MCITP Training at certkingdom.com

 

 

Tech salaries jump 5.3%, bonuses flat

Tech salaries jump 5.3%, bonuses flat
Tech and engineering pros reported the largest salary jump in more than a decade, according to career site Dice

Average salaries for tech pros climbed 5.3% to $85,619 last year, up from $81,327 in 2011. It’s the largest salary jump in more than a decade, according to career site Dice, which specializes in jobs for tech and engineering professionals.

Entry level talent (two years or less experience) waited three years to see an increase in average annual pay — and the market made up for the stagnancy with an 8% year-over-year increase to $46,315. At the other end of the spectrum, average salaries for tech professionals with at least 15 years of experience topped six-figures for the first time, growing 4% to $103,012.


2013 JOB WATCH: Top 11 metro areas for tech jobs

“Employers are recognizing and adjusting to the reality of a tight market,” said Scot Melland, CEO of Dice Holdings, in a statement. “The fact is you either pay to recruit or pay to retain and these days, at least for technology teams, companies are doing both.”

Tech bonuses were slightly more frequent — 33% of respondents got one in 2012 compared to 32% in 2011 — but slightly less lucrative at an average of $8,636 (down from $8,769). [Related story: “Outlook for IT bonus pay murky”]
tech salaries

“In the early stages of the recovery, companies were staying flexible by using performance pay to reward their top performers,” Melland said. “Now, companies are writing the checks that will stick. With a 3.8% tech unemployment rate, no one wants to lose talent.”

By location, Pittsburgh tech pros saw the largest salary increase, up 18% year/year to $76,207. Six other cities also reported double-digit growth in salaries — which is the most ever registered by the Dice Salary Survey.

San Diego (+13% to $97,328)
St. Louis (+13% to $81,245)
Phoenix (+12% to $83,607)
Cleveland (+11% to $75,773
Orlando (+10% to $81,583)
Milwaukee (+10% to $81,670)

Silicon Valley remains the only market where tech professionals average six-figure salaries ($101,278).

Across the U.S., big data skills are in demand, as evidenced by $100,000+ salaries for tech pros who use Hadoop, NoSQL and MongoDB. By comparison, average salaries associated with cloud and virtualization are just under $90,000 and mobile salaries are closer to $80,000, Dice reports.

“We’ve heard it’s a fad, heard it’s hyped and heard it’s fleeting, yet it’s clear that data professionals are in demand and well paid,” said Alice Hill, managing director of Dice.com. “Tech professionals who analyze large data streams and strategically impact the overall business goals of a firm have an opportunity to write their own ticket. The message to employers? If you have a talented data team, hold on tight or learn the consequences of a lift-out.”

Looking ahead to the current year, 64% of tech professionals are confident they could find a favorable new job in 2013.

Dice surveyed 15,049 employed tech professionals between Sept. 24 and Nov. 16, 2012, for its annual Salary Survey.


MCTS Training, MCITP Trainnig

Best Microsoft MCTS Certification, Microsoft MCITP Training at certkingdom.com