Blog: Barney Finucane Subscribe to this blog's RSS feed!

Barney Finucane

Welcome to my BeyeNETWORK blog. My main goal here is to address hype issues that come up in the Internet, not to provide any overview of the BI market itself. I look forward to any questions or comments you may have.

About the author >

Barney Finucane has extensive experience in the BI industry. As a consultant, he has supported companies in the chemical, energy and manufacturing sector with the introduction of BI software. As product manager for the company MIS, he was responsible for the front-end products Plain and onVision, and kept a keen eye on projects and tools from other vendors. His areas of speciality include tool selection, quality assurance for BI, data warehouse strategies and their architectures.

There has been quite a bit of discussion about whether Alphago can learn from the games it plays against Lee Sedol. I think not. At least, not directly. 
The heart of the program is the ”policy network” a convolutional neural network (CNN) that was designed for image processing. CNNs return a probability that a given image belongs to each of a predefined set of classifications, like ”cat”, ”horse”, etc. CNNs work astonishingly well, but have the weakness that they can only be used with a fix size image to estimate a fixed set of classifications.
The policy network views go positions as 19í—19 images and returns probabilities that human players would make one of 361 possible moves. This probability drives with the Monte Carlo tree search for good moves that has been used for some time in go computers.The policy network is trained on 30 million positions (or moves) initially. 
CNN (aka ”deep learning”) behavior is pretty well understood. The number of samples required for learning depends on the complexity of the model. A model of this complexity probably requires tes of thousands of example positions before it changes much. 
The number of samples required to train any machine learning program depends on the complexity of the strategy, not on the number of possible positions. For example, Gomoku ("five in a row", also called goban) on a 19í—19 board would take many fewer examples to train than go would, even though the number of possible positions is also very large.
Another point: Any machine learning algorithm will eventually hit a training limit, after which it won’t be able to improve itself by more training. After that, a new algorithm based on a new model of game play would be required to improve the play. It is interesting that the Alphago team seems to be actively seeking ideas in this area. Maybe that is because they are starting to  hit a limit, but maybe it's just because they are looking into the future.
So Alphago probably can’t improve its play measurably by playing any single player five times, no matter how strong. That would be ”overfitting”. The team will be learning from the comments of the pro players and modifying the program to improve it instead.
Interesting tidbit: Alphago said the chances of a human playing move 37 in game 2 was 1 in 10,000. So the policy network doesn’t decide everything.

Posted March 13, 2016 1:13 PM
Permalink | No Comments |
The key part of Alphago is a convolutional neural network. These are usually used for recognizing cat pictures and other visual tasks, and progress in the last five years has been incredible.
Alphago went from the level of a novice pro last October to world champion level for this match. It did so by playing itself over and over again.
Chess programs are well understood because they are programmed by humans. Alphago is uses an algorithm to pick a winning move in a given go position. But the heart of the program is a learning program to find that algorithm, not the algorithm itself.
Go programs made steady progress for a decade with improved tree pruning methods, which reduce the total number of positions the program has to evaluate. The cleverest method is Monte Carlo pruning, which simply prunes at random. 

Posted March 13, 2016 6:00 AM
Permalink | No Comments |

Paraccel was founded in 2005 by former Netezza executives as an alternative to Netezza's DW appliance idea. Paraccel runs on a commodity platform. It got lots of VC money (I've heard estimates as high as $90m) but has comparatively few customers. 

Recently Paraccel apparently hoped to generate a little revenue with hosting deals, which are always low priced. Or maybe they were just in it for the publicity. The technology is used for MicroStrategy Wisdom and I doubt MicroStrategy paid them very much. Shortly thereafter, they entered a deal with Amazon to license their technology for Redshift, which Amazon is reselling at very low rates.

Paraccel's struggles aren’t very surprising, since ALL the other vendors in the space except Teradata lost their independence in 2010/2011. Sybase, Netezza, Kickfire, Greenplum, Vertica and Aster Data were all acquired. Also HP killed Neoview in the same time frame. 

Actian is the newish name for Ingres, and controls the open source database by the same name. It also has several other databases including VectorWise, and there seems to me to be a good deal of overlap between the products. It is not a high profile company but it will be interesting to see what strategy they adopt to squeeze cash out of this highly funded and presumably expensive product.

Posted April 27, 2013 1:42 PM
Permalink | No Comments |
Have a look at this TIME article. The headline is "Americans Are Eating Fewer Calories, So Why Are We Still Obese?".

It goes on to cite two studies. One study shows that old people eat less fast food then young adults, and that in general food consumption is declining. This comes from the aging population I guess. The other shows a small decline (about 7%) in the calorie consumption of children. 

Neither study showed a decrease in calorie consumption by adult Americans, as far as I can tell. The naive assumption would be that consuming fewer calories would result in less obesity, but no data on the topic is presented. The headline is a complete misdirection.

The moral of the story is you should limit your observations to what the data actually says.

Posted March 27, 2013 1:56 PM
Permalink | No Comments |
Analysts like to provide a list of predictions for the coming year. I've never been a big fan because I think that real tipping points are pretty rare in any industry. People tend to overestimate the short-term change and underestimate long-term change.

Here are some trend ideas that are floating around the internet. There are basically three kinds:

New data sources: This mostly means Big Data, whatever that means. (Personally, I like Amazon's definition, which is "too much data to handle on a single server". Other definitions strive to include appliances like Teradata and HANA.)

New applications: Sentiment analysis, predictive analytics, collaboration

New UIs and functions: Better dashboards and visualizations, more self service and agility, voice interfaces, mobile.

New Platforms: This is mostly in memory, cloud and SaaS.

My take on all this is simple:

New data sources: I think big data is still mostly for online and mobile providers. It's true that manufacturers and retailers are trying to figure out how to make better use of the huge amounts of data their business directly and indirectly generates. But this business is still heavily dependent on boutique providers that bring a lot of domain knowledge and deep understanding of statistics with into the deal. I do not think it will have much impact on existing BI business. It's something different.

New applications: The same remarks apply to sentiment analysis as to big data. Predictive analytics is a more interesting market, but to find large, non-specialist audiences, vendors need to prove that their "black box" predictions are as reliable as the expertise of business users without explaining the math behind them.

New user interfaces and functions continue to appear, but I believe that as long as most BI companies stick to solving the easy problems, like making software look cool in a demo, and ignore the harder problems, like user-friendly data governance, there will be no big changes here. Mobile has surprised me, but it still hasn't made a big difference in the BI business.

New platforms. It is good to remember that business users don't care what platform is used, and that the most successful projects are controlled by business users. A platform is only good in the sense it delivers things like speed and convenience. It doesn't add any value per se.

Posted March 14, 2013 9:13 AM
Permalink | No Comments |
Have a look at this TIME article. The headline is "Americans Are Eating Fewer Calories, So Why Are We Still Obese?".

It goes on to cite two studies. One study shows that old people eat less fast food then young adults, and that in general food food consumption is declining. This comes from the aging population I guess. The other shows a small decline (about 7%) in the calorie consumption of children. 

Neither study showed a decrease in calorie consumption by adult Americans, as far as I can tell. The naive assumption would be that consuming fewer calories would result in less obesity, but no data on the topic is presented. The headline is a complete misdirection.

The moral of the story is you should limit your observations to what the data actually says.

Posted February 22, 2013 9:20 AM
Permalink | No Comments |
Christmas shopping this year really drove home to me how completely the electronics industry has made itself obsolete in recent years. The electronics stores are emptying out. All the gadgets that have been so popular in recent decades -- cameras, camcorders, VCRs, tape recorders, CD players, portable music boxes, dictation machines, game consoles, pagers, PCs, notebook computers, TVs, radios, pocket calculators, GPS navigation devices, synthesizers, mixing consoles and of course telephones (mobile and land based) have all disappeared into smart phones.

But smart phones aren't really phones at all, they are just palmtop computers that include an interface for cellular networks and a phone app. They are called smart phones for marketing reasons -- because the phone companies use them to lock consumers into overpriced network contracts. If photography were the killer app, they would be called smart cameras, which is just as appropriate. The only thing that seems to be keeping the entire electronics industry from being swallowed up by the black hole of Moore's Law is this kind of marketing wheeze and the (rapidly falling) price of screens.

I'm old enough to remember when Radio Shack was a national treasure. Now it's just a place to buy batteries while being harassed by a hard selling salesman from some telecoms oligopolist best known for hating its customers. And in this age of smart hearing aids, robotic assembly lines and fly-by-wire airplanes, its not just consumer electronics that is being computerized.

Posted December 30, 2011 2:17 PM
Permalink | No Comments |
There is a discussion of some remarks I made to Ann All at IT Business Edge here, together with some opnions by Howard Dresner.

The interview reflects my opinion pretty well. I see mobile BI as a way to find new types of customers for BI more than as a way to replace existing installations. Too bad I didn't mention salespeople on the road, I think they are an important potential market as well.

Another point is that I think most people who said they expected mobile BI to be in use within 12 months were being too optimistic. The BI Survey 11 will address this question.

Posted December 14, 2011 1:24 PM
Permalink | No Comments |
I'm talking to Tibco about Spotfire. It's an interesting product that I've reviewed before. It seems to me that they are moving more and more into operative BI, which fits the Tibco idea of fast data delivery that fits Tibco very well. They also seems to be putting more emphasis on ROLAP than they used to.

What is interesting is that they are also presenting a social media tool called Tibbr (wonder how they came up with that name!) and a cloud version. There's nothing wrong with this of course, but it seems to me that it doesn't fit their bus and/or ROLAP approach very well.

Their justification for the investment is that some analyst or other is predicting fast growth in this area. This reminds me of what an important ole analysts play in the market. Thanks to the analysts, a lot of BI vendors are jumping on the cloud bandwagon, even though the cloud sales channel is very different from what most BI vendors are accustomed to, and that the idea of mving data off site and then back onsite adds complexity.

Posted November 17, 2011 3:11 AM
Permalink | No Comments |
I'm talking to QlikTech about their new version, coming soon. I'll be publishing the results in the BI Verdict. They are focusing more and more on these bigger accounts. I think this story fits pretty well in my informal series of posts on the subject of agile BI. By coincidence I have already discussed QlikView in a previous post.

QlikTech has a what they call a "land and expand" policy, which means getting a single department on the tool and expanding from there. Actually, all BI companies that can deliver departmental solutions have something similar. The reason for this is simple: The cost of sales for selling to a company that is already using the tool is much lower than for completely new customers. In fact, a lot of BI tools spread through companies from department to department this way.

Now QlikTech is concentrating more on enterprise accounts. So it's interesting to see that the company is moving away from the previous claim that the tool is a replacement for a data warehouse. I think that any attempt on their part to compete as an enterprise solution would just distract them from their end users.

A lot of BI companies go through a similar life cycle as they grow. Most start out as ways to create departmental solutions, which tend to be faster, more agile projects. As they get bigger management tends to concentrate on larger accounts, which means making sure they are acceptable to the IT department. But IT is more interested in keeping processes running than in agile development. As a result, the products tend to become more complex and less suitable to agile solutions.

This is a big issue for QlikView right now because they have grown so quickly in recent years. But currently the company seems to backing away from radical changes in the tool. But it applies to and BI tool that is growing.

Posted August 18, 2011 3:37 AM
Permalink | No Comments |