How will "Deep Learning" change our daily lives in 2016?

navigation-1048294_640.jpg

“Deep Learning” is one of the major technologies of artificial intelligence.  In April 2013, two and half years ago, MIT technology review selected “Deep Learning” as one of the 10 breakthrough technologies 2013.  Since then it has been developed so rapidly that it is not a dream anymore now.   This article is the final one in 2015.  Therefore, I would like to look back the progress of “Deep Learning” this year and consider how it changes our daily lives in 2016.

 

How  has “Deep Learning” progressed in 2015? 

1.  “Deep Learning” moves from laboratories to software developers in the real world

In 2014,  Major breakthrough of deep learning occurred in the major laboratory of big IT companies and universities. Because it required complex programming and huge computational resources.  To do that effectively, massive computational assets and many machine learning researchers were required.  But in 2015,  many programs, softwares of deep learning jumped out of the laboratory into the real world.  TorchChainerH2O and TensorFlow are the examples of them.  Anyone can develop apps with these softwares as they are open-source. They are also expected to use in production. For example, H2O can generate the models to POJO (Plain Old Java Code) automatically. This code can be implemented into production system. Therefore, there are fewer barriers between development and production anymore.  It will accelerate the development of apps in practice.

 

2. “Deep Learning” start understanding languages gradually.

Most of people use more than one social network, such as Facebook, Linkedin, twitter and Instagram. There are many text format data in them.  They must be treasury if we can understand what they say immediately. In reality, there are too much data for people to read them one by one.  Then the question comes.  Can computers read text data instead of us?  Many top researchers are challenging this area. It is sometimes called “Natural Language Processing“.  In short sentences, computers can understand the meaning of sentences now. This app already appeared in the late of 2015.  This is “Smart Reply” by  Google.  It can generate candidates of a reply based on the text in a receiving mail. Behind this app,  “LSTM (Long short term memory)” which is one of the deep learning algorithm is used.  In 2016, computers might understand longer sentences/paragraphs and answer questions based on their understanding. It means that computers can step closer to us in our daily lives.

 

3. Cloud services support “Deep Learning” effectively.

Once big data are obtained,  infrastructures, such as computational resources, storages, network are needed. If we want to try deep learning,  it is better to have fast computational resources, such as Spark.  Amazon web services, Microsoft Azure, Google Cloud Platform and IBM Bluemix provide us many services to implement deep learning with scale. Therefore, it is getting much easier to start implementing “Deep Learning” in the system. Most cloud services are “pay as you go” so there is no need to pay the initial front cost to start these services. It is good, especially for small companies and startups as they usually have only limited budgets for infrastructures.

 

 

How will “Deep Learning” change our daily lives in 2016? 

Based on the development of “Deep learning” in 2015,  many consumer apps with “Deep learning” might appear in the market in 2016.   The deference between consumer apps with and without “Deep Learning” is ” Apps can behave differently by users and conditions”. For example,  you and your colleagues might see a completely different home screen even though  you and your colleagues use the same app because “Deep learning” enables the app to optimize itself to maximize customer satisfaction.  In apps of retail shops,  top pages can be different by customers according to customer preferences. In apps of education,  learners can see different contents and questions as they have progressed in the courses.  In apps of navigations,  the path might be automatically appeared based on your specific schedule, such as the path going airport on the day of the business trip.  They are just examples.  It can be applied across the industries.  In addition to that,  it can be more sophisticated and accurate if you continue to use the same app  because it can learn your behavior rapidly.  It can always be updated to maximize customer satisfactions.  It means that we do not need to choose what we want, one by one because computers do that instead of us.  Buttons and navigators are less needed in such apps.  All you have to do is to input the latest schedules in your computers.  Everything can be optimized based on the updated information.  People are getting lazy?  Maybe yes if apps are getting more sophisticated as expected. It must be good for all of us.  We may be free to do what we want!

 

 

Actually,  I quit an investment bank in Tokyo to set up my start-up at the same time when MIT  technology review released 10 breakthrough technologies 2013.  Initially I knew the word “Deep Learning” but I could not understand how important is is to us because it was completely new for me. However, I am so confident now that I always say “Deep Learning'” is changing the landscape of jobs, industries and societies.  Could you agree with that?  I imagine everyone can agree that by the end of 2016!

 

 

 

Notice: TOSHI STATS SDN. BHD. and I do not accept any responsibility or liability for loss or damage occasioned to any person or property through using materials, instructions, methods, algorithm or ideas contained herein, or acting or refraining from acting as a result of such use. TOSHI STATS SDN. BHD. and I expressly disclaim all implied warranties, including merchantability or fitness for any particular purpose. There will be no duty on TOSHI STATS SDN. BHD. and me to correct any errors or defects in the codes and the software.

These are small Christmas presents for you. Thanks for your support this year!

I started the group of “big data and digital economy” in Linked in on 15th April this year. Now the participants are over 300 people!  This is beyond my initial expectation. So I would like to appreciate all of you for your support.

I prepare several small Chirstmas presents here. If you are interested in, please let me know. I will do my best!

 

1. Your theme of my weekly letter

As you know, I write the weekly letter “big data and digital economy” every week and publish it in Linkedin. If you are interested in specific themes,  I would like to research and write them as long as I can. Anything is OK if it is about digital economy.  Please let me know!

 

2.  Applications of data analysis in 2016

In 2016,  I would like to develop my applications using data analysis and make them public through the internet.  As long as data is “public”,  we can do any analysis on the data. Therefore,  if you would like to look at your own analysis based on public data,  could you let me know what you are interested in?    These are examples of applications provided by “shiny”,  very famous tool among data scientists.

http://shiny.rstudio.com/gallery/

 

3.   Announcement on the  project of R-programming platform

This is a project of my company in 2016.  To support for business personnel to learn R-programming,  I would like to set up the platform where participants can learn R-programming interactively with ease.  Contents are very important in order for participants to keep learning motivations. When you have specific themes which you want to learn,  could you let me know?  These themes may be included as programs in the platform going forward!    This is an introductory video of the platform.

http://www.toshistats.net/r-programming-platform/

 

Thanks for your support in 2015 and let us enjoy predictive analytics in 2016!

Do we need "snow" to cerebrate Christmas in December?

There are many Christmas trees in shopping malls.  It makes us a little happier.  Children must expect big presents at Christmas eve.   I am also waiting for my presents, although I do not  know where my Santa Claus is now.

This is the second time when I live in KL at the time of Christmas.  Then I feel a little strange because it is hot at time of  Christmas season in KL.  In Japan,  it is cold and it has sometimes massive snow in December. Whenever I saw Christmas trees in Japan, it was always cold.  But now it is hot in KL!  I think  most of Asean countries have no snow so there are few opportunities where we can feel “snow”.

The picture above is taken in KL. On the roof of the house, there is snow. But I do not see snow on the trees.  White balls look just decorations for me.   It must be OK as there is no snow in KL.  On the other hand,  this picture below is taken in Japan.  There are many symbols of snow on the Christmas trees.


Some of you have been to Hokkaido, the north part of Japan to enjoy snow in winter.  The whole land is sometimes covered with “snow” there in winter.  So everything looks white and it is very quiet, no sound is heard  because noises are absorbed by thick snow on the ground. In a such case, Christmas trees must have “snow” on them.  So it may be different,  location by location.

I do not have any statistics of ” how many trees have ‘snow” on them in shopping malls all over the world”. But it is interesting for me because it tells us how weather and climate affect our behaviors. Because Japan has four seasons (spring, summer, autumn and winter),  predictions of its climate are very important for companies as well.  Hotter summer means more sales of juice, ice cream and air conditioners, vice versa.  If winter is not so cold than usual,  sweaters and coats are not selling well. It means less flu so it is good for children and senior people, but it is not so good for the pharmaceutical industry. In this way, weather and climate have huge impacts to our behavior and economy.

The data about weather and climate may be relatively unused in companies in order to make business decisions so far.  But as we have more data about them and obtain predictions with accuracy, it is worthwhile using data about weather and climate in the businesses now. I would like to take examples of analysis about weather and climate going forward.


Anyway,  Merry Christmas for all of you!

Can computers write sentences of docs to support you in the future?

This is amazing!  It is one of the most incredible applications for me this year!  I am very excited about that.  Let me share with you as you can use it,  too.

This is "Smart Reply of Inbox", an e-mail application from Google.  It was announced on 3rd November. I try it today.

For example, I got e-mail from Hiro. He asked me to have a lunch tomorrow. In the screen, three candidates of my answer appear automatically.  1. Yes, what time?  2. Yes, what's up  3. No, sorry.  These candidates  are created after computers understand what Hiro said in the e-mail. So each of them is very natural for me.

 

So all I have to do is just to choose the first candidate and send it to Hiro.  It is easy!


According to Google, state of the art technology "Long short term memory" is used in this application.

I always wonder how computers understand the meaning of words and sentences.  In this application, sentences are represented in fixed sized vectors. It means that each sentence is converted to sequences of numbers.  If two sentences have the same meaning,  the vector of each sentence should be similar to each other even though the original sentences look different.


This technology is one of the machine learning. Therefore,  the more people use it, the more sophisticated it can be because it can learn by itself.  Now it applies to relatively short sentences like e-mail. But I am sure it will be applied to longer sentences, such as official documents in business.  I wonder when it happens in the future.  Pro. Geoffrey Hinton is expected to research this area with intense.  If it happens, computers will be able to understand what documents mean and create some sentences based on their understanding.  I do not know how Industires are changed when it happens.

This kind of technology is sometimes referred as "Natural language processing" or "NLP".   I want to focus on this area as a main research topic of my company in 2016.  Some progresses will be shared through my weekly letter here.


I would like to recommend you to try Smart Reply of Inbox and enjoy it!  Let me know your impressions. Cheers!




Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I and TOSHI STATS.SDN.BHD. accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

"Community" accelerates the progress of machine learning all over the world!

When you start learning programming,  it is recommended to visit the sites of community of languages.  "R" and "python" have big communities, and they have been contributing to the progress of each language. This is good for all users. H2O. ai also held an annual community conference "H2O WORLD 2015"  this month.  Now video and presentation slides are available through the internet. I could not attend the conference as it was held in Silicon Valley in the US. But I can follow and enjoy it just by going through websites. I recommend you to have a quick look to understand how knowledge and experiences can be shared at the conference. It is good for anyone who are interested in data analysis.

 

1.  The user communities can accelerate the progress of open source languages

When I started learning "MATLAB®" in 2001,  there were few user communities in Japan as far as I knew.  So I should attend the paid seminars to learn this language, which were not cheap.  But now most of uses communities are available without any fee. In addition to that,  this kind of communities have been bigger and bigger recently.   One of the main reasons is that number of "open source languages" are increasing recently.    "R" and "python" are also open source languages. It means that when someone want to try certain language,  all they have to do is just "downloadand use it".  Therefore, users can be increased at an astonishing pace.  On the other hand,  if someone want to try "proprietary software" such as MATLAB, they must buy each license before using it. I loved MATLAB for many years and recommended my friends to use it. But unfortunately no one uses it privately because it is difficult to pay license fee privately.  I imagine that most users of proprietary software are in organizations such as companies and universities.  In such case, organizations pay license fees.  So each individual can enjoy no freedom to choose languages they want to use. Generally it is difficult to switch from one language to another when proprietary softwares are used. It is called "Vendor lock-in".  Open source languages can avoid that. This is one of the reasons why I love open source languages now. The more people can use, the more progress can be achieved.  New technologies such as "machine learning" can be developed through user communities because more users will join going forward.

 

2.  The real industry experiences can be shared in communities

It is the most exciting part of the community.  As a lot of data scientists and engineers from industry join communities,  their knowledge and experience are shared frequently.  It is difficult to find this kind of information in other places.  For example, the theory of algorithms and methods of programming can be found in the courses provided by universities in MOOCs. But there are few about industry experiences in MOOCs in a real time basis.  For example, in H2O WORLD 2015,  there are sessions with many professionals and CEOs from industries. They share their knowledge and experiences there.  It is a treasure not only for experts of data analysis, but for business personnel who are interested in data analysis. I would like to share my own experience in user communities in future.

 

3.  Big companies are supporting user communities

Recently major IT big companies have noticed the importance of the user community and try to support them.  For example, Microsoft supports "R Consortium" as a platinum member. Google and Facebook support communities of their open source languages, such as "TensorFlow" and "Torch".  Because new things are likely to happen and be developed among users outside the companies.  Therefore It is also beneficial to big IT companies when they support user communities. Many other IT companies are supporting communities, too. You can find many names as sponsors under the big conference of user communities.

 

The next big conference of user communities is "useR! - International R User Conference 2016".  It will be held on June 2016.  Why don't you join us?  You may find a lof of things there. It must be exciting!

 

 

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I and TOSHI STATS.SDN.BHD. accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

"Speed" is the first priority of data analysis in the age of big data

When I learned data analysis a long time ago,  the number of samples of data was from 100 to 1,000. Because teachers should explain what the data are in the details.  There were  a little parameters that was calculated, too.  Therefore, most of statistical tools could handle these data within a reasonable time.  Even spread sheets worked well.  There are huge volume data,  however,  and there are more than 1,000 or10,000 parameters that should be calculated now.  We have problems to analyze data because It takes too long to complete the analysis and obtain the results.  This is the problem in the age of big data.

This is one of the biggest reasons why new generation tools and languages of machine learning appear in the market.  Torch became open sourced from Facebook at January 2015.  H2O 3.0 was released as open source in May 2015 and TensorFlow was also released from Google as open source in this month.  Each language explains itself as "very fast" language.

 

Let us consider each of the latest languages.  I think each language puts importance into the speed of calculations.  Torch uses LuaJIT+C, H2O uses Jave behind it.  TensorFlow uses C++. LuaJIT , Java and C++ are usually much faster compared to script languages such as python or R. Therefore new generation languages must be faster when big data should be analyzed.

Last week, I mentioned deep learning by R+H2O.  Then let me check how fast H2O runs models to complete the analysis.  This time, I use H2O FLOW,  an awesome GUI,  shown below.  The deep learning model runs on my MAC Air11  (1.4 GHz Intel Core i5, 4GB memory, 121GB HD) as usual.  Summary of the data used  as follows

  • Data: MNIST  hand-written digits
  • Training set : 19000 samples with 785 columns
  • Test set : 10000 samples with 785 columns

Then I create the deep learning model with three hidden layers and corresponding units (1024,1024,2048).  You can see it in red box here. It is a kind of complex model as it has three layers.

 

It took just 20 minutes to complete. It is amazing!  It is very fast, despite the fact that  deep learning requires many calculations to develop the model.  If deep learning models can be developed within 30 minutes,  we can try many models at different setting of parameters to understand what the data means and obtain insight from them.


I did not stop running the model before it fitted the data.  These confusion matrices tell us error rate is 2.04 % for training data (red box) and 3.19 % of test data (blue box). It looks good in term of  data fitting.  It means that 20 minutes is enough to create good models in this case.



Now it is almost impossible to understand data by just looking at them carefully because it is too big to look at with our eye. However,  through analytic models, we can understand what data means. The faster analyses can be completed,  the more  insight can be obtained from data. It is wonderful for all of us.  Yes, we can have an enough time to enjoy coffee and cakes with relaxing after our analyses are completed!



Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I and TOSHI STATS.SDN.BHD. accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

This is my first "Deep learning" with "R+H2O". It is beyond my expectation!

Last Sunday,  I tried “deep learning” in H2O because I need this method of analysis in many cases. H2O can be called from R so it is easy to integrate H2O into R. The result is completely beyond my expectation. Let me see in detail now!

 

1. Data

Data used in the analysis is ” The MNIST database of handwritten digits”. It is well known by data-scientists because it is frequently used to validate statistical model performance.  Handwritten digits look like that (1).

 

Each row of the data contains the 28^2 =784 raw grayscale pixel values from 0 to 255 of the digitized digits (0 to 9). The original data set of The MNIST is as follows.

  • Training set of 60,000 examples,
  • Test set of 10,000 examples.
  • Number of features is 784 (28*28 pixel)

The data in this analysis can be obtained from the website (Training set of 19,000 examples, Test set of 10,000 examples).

 

 

2. Developing models

Statistical models learn by using training set and predict what each digit is by using test set.  The error rate is obtained as “number of wrong predictions /10,000″. The world record is ” 0.83%”  for models without convolutional layers, data augmentation (distortions) or unsupervised pre-training (2). It means that the model has only 83 error predictions in 10,000 samples.

This is an image of RStudio, IDE of R.  I called H2O from R and write code “h2o.deeplearning( )”.  The detail is shown in the blue box below.  I developed the model with 2 layers and 50 size for each. The error rate is 15.29% (in the red box).  I need more improvement of the model.

Then I increase the number of layers and sizes.  This time,   I developed the model with 3 layers and 1024, 1024, 2048 size for each. The error rate is 3.22%, much better than before (in the red box).  It took about 23 minutes to be completed. So there is no need to use more high-power machines or clusters so far ( I use only my MAC Air 11 in this analysis). I think I can improve the model more if I tune parameters carefully.

Usually,  Deep learning programming is a little complicated. But H2O enable us to use deep learning without programming when graphic user interface “H2O FLOW” is used.  When you would like to use R,   the command of deep learning to call H2Ois similar to the commands for linear model (lm) or generalized linear model (glm) in R. Therefore, it is easy to use H2O with R.

 

 

This is my first deep learning with R+H2O. I found that it could be used for a variety cases of data analysis. When I cannot be satisfied with traditional methods, such as logistic regression, I can use deep learning without difficulties. Although it needs  a little parameter tuning such as number of layers and sizes,  it might bring better results as I said in my experiment. I would like to try “R+H2O” in Kaggle competitions, where many experts compete for the best result of predictive analytics.

 

P.S.

The strongest competitor to H2O appears on 9 Nov 2015.  This is ” TensorFlow” from Google.  Next week,  I will report this open source software.

 

Source

1. The image from GitHub  cazala/mnist

https://github.com/cazala/mnist

2. The Definitive Performance Tuning Guide for H2O Deep Learning , Arno Candel, February 26, 2015

http://h2o.ai/blog/2015/02/deep-learning-performance/

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I and TOSHI STATS.SDN.BHD. accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

"H2O", this is an awesome tool of "Digital marketing" for everyone!

Last week I found the awesome tool for digital marketing as well as data analysis.  It is called “H2O“.  Although it is open source software, its performance is incredible and easy to use.  I would like to introduce it to Sales/Marketing personnel who are interested in Digital marketing.

“H2O is open-source software for big-data analysis. It is produced by the start-up H2O.ai(formerly 0xdata), which launched in 2011 in Silicon Valley. The speed and flexibility of H2O allow users to fit hundreds or thousands of potential models as part of discovering patterns in data. With H2O, users can throw models at data to find usable information, allowing H2O to discover patterns. Using H2O, Cisco estimates each month 20 thousand models of its customers’ propensities to buy while Google fits different models for each client according to the time of day.” according to Wikipedia(1).

Although its performance looks very good, it is open source software. It means that everyone can use the awesome tool without any fee.  It is incredible!  “H2O” is awarded one of ” Bossie Awards 2015: The best open source big data tools” (2).  This image shows H2O user interface “H2O FLOW”.

By using this interface, you can use the state of art algorithm such as “Deep learning” without programming.  It is very important for beginners of data analysis. Because they can start data analysis without programming anyway.  Dr. Arno Candel,   Physicist & Hacker at H2O.ai. , said  “And the best thing is that the user doesn’t need to know anything about Neural Networks”(3).  Once models are developed by this user interface, program of the model with “Java” is automatically generated.  It can be used in production systems with ease.

 

 

One of the advantages of open source is that many user’s cases are publicly available. Open source can be public, therefore it is easy to be distributed as users’ experiences of “What is good?” and “What is bad?”.   This image is a collection of tutorials “H2O University“.  It is also available for free. There are many other presentations, videos about H2O in the internet, too! You may find your industry”s cases among them. Therefore, there is a lot of materials to learn H2O by ourselves.

 

In addition to that,  “H2O” can be used as an extension of “R“.  R is one of the most widely-used analytical language.  “H2O” can be controlled from R console easily. Therefore“H2O” can be integrated with R.  “H2O” also can be used with Python.

There are so many other functionalities in H2O. I cannot write everything here.  I am sure it is an awesome tool for both business personnel and data scientists.  I  would like to start using “H2O” and publish my experiences of “H2O”going forward. Why don’t you join “H2O community”?

 

 

Source

1.Wikipedia:H2O (software)

https://en.wikipedia.org/wiki/H2O_(software)

2.Bossie Awards 2015: The best open source big data tools

http://www.infoworld.com/article/2982429/open-source-tools/bossie-awards-2015-the-best-open-source-big-data-tools.html#slide4

3.Interview: Arno Candel, H2O.ai on the Basics of Deep Learning to Get You Started

http://www.kdnuggets.com/2015/01/interview-arno-candel-0xdata-deep-learning.html

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I and TOSHI STATS.SDN.BHD. accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

Do you know this monumental building for peace in Japan?

This is the last week of my business trip to Japan. So I went to Hiroshima which is in the west part of Japan. Because I would like to look at one of the most famous old buildings in Japan.

This is called "Hiroshima Peace Memorial" ( In short, Genbaku dome in Japanese).  As  you know,  Hiroshima city was attacked by atomic bombing by US army in August 1945 and more than 100,000 people were killed.  This building could survive under the attack of this bombing.  Therefore, in 1996, the Genbaku dome was registered on the UNESCO World Heritage List based on the Convention for the Protection of the World Cultural and Natural Heritage. Many foreigners come here and see what was going on there in 1945.  I am very glad for them to come to Hiroshima and share our thoughts with them.

 

It is good for many people from abroad to come here and think about peace of the world.  The problem is Hiroshima-city is located far from Tokyo, center of Japan.  Many people use "Shinkansen" or "Bullet train" to go from Tokyo to Hiroshima as it is very convenient.  It usually, however, costs more than 18,000 JPY (around 600MRG).  Some foreign tourists may give up going to Hiroshima because it costs too much.

To solve this problem, I would like to recommend using domestic flights from Narita to Hiroshima.  In my case, it costs around 8,000 JPY (6,000 JPY for a domestic flight,   around 2,000 JPY for access from cities to airports), although it takes a little more time compared with Shinkansen totally.  This flight is operated by Spring Japan.  I would like to recommend this flight for everyone as it is smooth and comfortable. This picture is the counter of Spring Japan in Narita airport terminal3.

Most of foreigners arrive at Haneda or Narita international airport in Japan.  I think It is a good choice to visit Hiroshima after visiting Tokyo.  There are some international airports in the west part of Japan.  The biggest one is Kansai international airport in Osaka.  Hiroshima also has its international airport.  So they can go back to their home countries from these airports after visiting Hiroshima.

 

When I have a business trip to Japan,  I usually arrive at Haneda international airport in Tokyo and leave Japan from Kansai international airport because I should move around the west part of Japan during my business trip.  I used only Shinkansen to go from Tokyo to cities in the west part of Japan before.  However, as domestic flights are increasing, especially flights operated by low cost carriers are increasing dramatically, now that we can have a lot of choices going around Japan. Why don"t you use domestic flights and enjoy more in Japan, too?

 

This is an additional information about convenience stores in Japan. Now there are many convenience stores inside ticket gates at stations. So you can enjoy shopping there just before getting on the train!

 

OK, I am going back to Kuala Lumpur tomorrow.  See you there again!

Who will be the winner in the competition of convenience stores in Japan?

030cc0d63c6ae43226f003c8b0bb4527_s.jpg

Today I am in Tokyo, Japan. So I would like to write convenience stores in Japan.  Since 7-eleven opened in 1974, there are a lot of varieties of  convenience stores in Japan. But as the populations of Japan are decreasing  and too many convenience stores are already in Japan,  Competions of convenience stores are getting tougher and tougher. Therefore, small and medium sized store chains have a hard time and some of them are acquired d by big  convenience store chains.  Now it is almost clear that the big three, which are 7-elevenLawson and Family Mart, dominate the market.

There are many convenience stores in Japan.  Therefore, they have huge impacts to merchandise there. For example,  a cup of coffee is served at the counter of most of convenience stores. Some of them are self-serviced. The taste is very good, although it is reasonable (around 100JPY).  They are getting popular and compete with canned  coffee in vendor machines or coffee shops.  You can try this coffee when you come to Japan as there are many convenience stores near the stations.

 

In terms of usage of big data for business decisions,  I am very interested in Lawson because it analyses data from stores and predict what products are popular.  This picture is my “Ponta CARD”.  When I buy products at Lawson, I present it at the counter.  So Lawson knows what and when I buy there.  It works all over Japan, therefore a huge amount of data is collected and analyzed everyday.

According to “Top Management Message October 7, 2015” by Lawson,  it introduces more advanced “semi-automatic ordering system”. Let see what it is.

“We began introducing our new semi-automatic ordering system from June to improve the delivery of products to our stores. The system is designed to recommend the most appropriate product lineup and number of items for delivery based on a range of data for ready-made snack meals and other categories, such as Ponta member purchasing trends, a store’s most recent sales data and information on heavy user purchases, information from other stores with a similar customer base, the weather, and finally information on the various campaigns conducted. The semi-automatic ordering system had been introduced in approximately 7,500 stores at the end of August 2015.” (1)

It is amazing!  Convenience store is usually not so big. Therefore, it is very important to know how many and what products are on store shelves. Data tells us how to do that accurately! I would like to research  what important factors are in this analysis going forward. You may be interested in them, too!

 

When you come to Japan,  you can find convenience stores at the every corner of the cities. There are many onigiri (rice ball), bento (lunch box), breads, beverages and sweets. Most of them are open 24 hours a day so you can enjoy shopping  anytime you want. Let us go there and see who will be the winner in the competition of convenience stores in Japan!

 

Source

1. Lawson website

http://lawson.jp/en/ir/message/backnumber/151007.html

 

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I and TOSHI STATS.SDN.BHD. accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

Do you know how "the community bus system" works in Japan?

This is a picture of Shin Urayasu station and it's bus terminal.  This week, I stay in Shin-Urayasu near Tokyo Disneyland as I have a business trip to Japan.   It is nice to stay here because the weather is always good.   So I walked outside and found small buses are going into the town.  It is called "community bus".  Please compare a white bus in the left and a red bus in the right on the picture above.  A left white bus is smaller than a red one.  Let me explain how this community bus works as the important part of the city transportation system.

 

In Japan,  most stations in cities have bus terminals in them. Just like the pictures above, many people can go to the stations from their home by using  buses. Some of them live in places where they cannot walk to stations as it takes too much time.  Therefore, buses are needed for people in cities to reach stations.

The problem is that buses are usually too big to drive through small roads in cities.  Buses usually run through main streets and people should go to  bus-stops along main streets.  It takes time to go from home to bus-stops along main streets.  Unlike other countries in Asia, Japan is an aging society.  So many senior people live here. It is very difficult for seniors to walk from home to bus stops along main streets.

This is why  "community bus" are introduced.  This picture shows us  "community bus".

Please compare a motorcycle to the bus.  You can find this bus is not so big.  Therefore the bus can drive along small roads in residential areas.  Bus stops can be located near homes.  It is good for senior people as they can go to stations, shopping malls and hospitals by "community bus" more easily.  It runs every 20 minutes in Urayasu city.

 

When you look at the picture above, you can notice that door steps are located lower than usual. Therefore It is easier for older people to get on and off "community bus" than big buses. Of course, it is more energy-efficient to bring a smaller number of people to destinations as it is lighter than a big bus.

 

Now many Asian cities including Kuala Lumpur have developed railway systems in cities to reduce traffic jams.  We also should consider, however, how we can go to and come back from the stations. Otherwise, railway systems cannot be used as main transportation systems because most people may continue to use cars.  In the future, with GPS system, we might be noticed that how we can use buses to go to destinations in a real-time basis through mobile devices. It must be more convenient as we do not need to wait buses at bus-stops for a long time!

What will be the flight service in the future? I write it in the air!

Now I am in the air from Kuala Lumpur to Tokyo as I have a business trip.  I always use Air Asia because it is convenient and reasonable. Since AirAsia has operated,  it is getting cheaper to flight from Kuala Lumpur to Tokyo. It is very good, especially for younger generations. I would like to welcome them in Japan very much.  Then I am wondering what the flight service will be in the future. Let us consider it with me!

 

1. Service on flight

Low cost carriers, including AirAisa increase the number of customers per flight compared with legacy carriers to reduce the price of the flight. Therefore services for each customer are not the sane as legacy carriers.  I think, however, it will be improved dramatically supported by digital technologies.  At each site, electronic dashboard might be equipped and all information, such as flight schedules,  emergency evacuation methods might be provided.  These are translated into many languages with machine translations so there is no need to worry about language barriers. ( In my flight of AirAsia, English, Japanese and Malay are used in the flight announcement. ) Meals in a fight will be improved, too.  We might order meals on demand through the electronic dashboard whenever you want to eat. These data can be collected customer by customer.  Therefore, preference of each customer might be known in advance.  This technology is called “personalization”. So low cost carriers might predict what kind of meals are needed in the flight based on past experience ofeach customer. It enables them to widen the variety of meals served because there is less risk to have a lack of inventories of meals on the flight.  To serve meals to each customer,  robots of cabin attendant assistants might support cabin attendants so that meals are served smoothly. I am excited if I can choose many varieties of meals on demand.

 

2. Immigration

Before getting on the board,  it takes time to pass immigration.  I always think it might be more effective with technologies called “face recognition”.  Computers can identify who you are by comparing to your face image stored on the passport. It is good to take less time to pass immigration for everyone. If it is connected to a database of INTERPOL,  it can enhance identification of criminals.

 

3.  Maintenance

Airplanes have a massive amount of parts. Therefore, maintenance is critically important to keep flights safe. Especially for low cost carriers, there is less time to maintain airplanes from landing to taking off again.  It can be enhanced by technologies   called “internet of things” and “predictive analytics“.  In internet of things,  each part has sensors and provide data periodically thought the internet.  Data from the sensors are collected and analyzed by “predictive analytics” to predict which parts are likely to fail in  advance.  Maintenance can be  more effective by using the results of  predictive analytics. Data from sensors can be transmitted from airlines to airports, even though they are in the air. Therefore failed parts or potential one can be identified before air plains land.  It enables us to decrease the time of maintenance.

 

Beyond low cost carriers,  the airplane in the air might be connected to other industries such as hotels. For example, the flight might be delayed due to bad weather and customers need reservations of hotels as the flight will land at the midnight. In such case, We can reserve hotels thought digital dashboard of each sheet. It is good to have reservations of the hotel even if we are in the air!

I hope my flights will be more comfortable in the future!  Could you agree?

 

 

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I and TOSHI STATS.SDN.BHD. accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

What will be a shopping mall in the future? It might be a community.

I like to go to shopping malls when I have a spare time.  There are restaurants, supermarkets, retail clothing, electrical equipment shops,  book stores, movie theaters, and so on.  Then I am wondering what will be a shape of shopping malls in the future.

 

1. Why do we need “real” shopping malls?

When we are busy and have no time to go shopping,  we would like to use e-commerce, such as Amazon.com.  So why should we go to a shopping mall?  The reason is that we would like to enjoy walking in big malls, seeing products, touching them physically and meeting friends there.  Shopping malls are good places to meet people as they are air-conditioned, located at convenient places such as near stations, have big parking lots.  In order to keep attracting people to shopping malls,  it should motivate people to come there.  Strategies might be different from countries.  In developing countries such as ASEAN countries,  new facilities and malls might be good motivators for people to go there. On the other hand, in developed countries such as Japan, shopping malls are in tough competitions each other.  It is getting harder to survive in the competition.  Populations of some of developed countries including Japan and Korea are decreasing gradually. Therefore, it is difficult to keep the number of people to come to shopping malls in the long run.  Only new facilities are not enough to compete in  developing countries’ markets.

 

2. How can the frequency of visiting shopping malls be increased?

The more frequently people are coming to the mall, the more people purchase products there. The problem is how people can be motivated to come to shopping malls consistently in the long period. I think a kind of mechanism should be introduced.  In order to encourage people to come to shopping malls,  many events are needed.  For example, amateur band performance contests,  kids programming contests,  digital art museums,  touring experience corners.  Digital technologies should be used at every event.  Amateur band performances might be broadcasted lively through the internet to mobile applications of members of shopping malls.  Kids programming contests can be held among several shopping malls by connecting each of them by the internet.  3D digital arts can be projected at super-high resolution level.  Virtual Reality enables us experience to go anywhere in the world.  These are just examples. There might be many innovations in the future. By supporting digital technologies,  Events could be held in parallel with  other events in a big shopping mall.  Each floor can have different events to motivate people to come there.  Therefore, combinations of many events should be optimized so that more people can enjoy them. I would like to call it  “Event portfolio management”.  To maximize the number of people coming to shopping malls, the best combinations of events should be identified. Costs, spaces and time allocations should be optimized to do that.  This is done by “Event portfolio optimization” with digital technologies.

 

3.  New services will emerge to support older people and busy mothers

When you do not come to shopping malls by car,  it is difficult to buy a lot of heavy products and bring them to your house.  In such case,  drones might bring them to your house instead of you. It is a good service for me because I do not drive a car in KL.  I do not need to worry about heavy products and continue to walk and enjoy events in shopping malls. This might be good! When I am getting older, it is almost impossible to bring heavy products by myself so I definitely need this service!  In addition to that,  young mothers may have a hard time to go shopping with many kids. Therefore, small entertainment facilities for kids might be needed. Mothers can take their kids there and go shopping alone.  These facilities will take care kids when Mothers are shopping.  There are a lot of virtual “puppets” so kids cannot be boring.  Kids’ behavior can be monitored and transmitted to Mother’s mobile devices so that mothers can be noticed what their kids are doing.  Everyone can be happy there!

 

Shopping malls are not just hardware.  These are communities for people to gather,  enjoy,  be satisfied with.  Therefore new entertainments and innovations are always needed going forward.  Supported by digital technologies,  they might be changed to satisfy us more and more. It must be exciting, isn’t it?

 

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I and TOSHI STATS.SDN.BHD. accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

Salesforce and Microsoft. Is the team a game changer of digital marketing?

Since Salesforce announced partnership with Microsoft® in May 2014 (1),  there might be a lot of rumors about this team.  Dreamforce in 2015 was held in San Francisco in the U.S on 15-18 September.  I found that this partnership is progressing rapidly.  As you  know, Salesforce is a king of CRM and Microsoft dominates the enterprise software market. Most of us use MS office365™ in our daily life.  Let us consider why this team might be a game changer in digital marketing.

 

1. One to one engagement

Salesforse said "How can you create One to one engagement with your customers? (2) There must be many answers. In my view we need to know what customers want more deeply in a real-time basis, then we should react it. To do that, we should have a mechanism to process massive amount data from customers effectively. I want to call it "Front, Back and Middle" mechanism. Front should face customers directly and collect data such as e-mails, phone calls, click-logs on mobile devices, purchases of products, payments, claims, and so on.  Back should record and store the data collected from front into storages or database.  Middle should analyze the data and provide insights to front so that front can make better business decisions. These processes are recurrent an they should be done many times in a seamless manner.  Sales personnel can obtain information and insights from this mechanism in real time-basis and face each customer one to one basis. In my view, that is "one to one engagement".

 

2. The combination of strong Front and Strong Middle/Back

Salesforce(SF) is a king of CRM. It means that SF is the strong front.  Microsoft (MS) expands its PaaS,  MS AZURE™ aggressively.  MS AZURE™ has a function of machine learning called AZURE ML . In MS AZURE™, there are many choices of database. Therefore, MS has strong middle and back.  Users can enjoy this strong combination of "front, middle and back" as the partnership between SF and MS is deepened recently.  I hope I can choose many functions from SF/ MS and set up systems based on my own preferences  in the future.   In my view,  this combination might be better than combinations between other big IT companies as corporate culture of SF/MS seems to be similar each other.  Since Satya Nadella became CEO of MS in Feb 2014,  MS culture seems to be changed from a traditional software company to a startup-minded cloud company.

 

3. More choices for users

PaaS can be used independently. Technologies are developing so fast, however, it seems to be difficult, that only one company covers everything to satisfy users' needs. Therefore, partnerships like SF/MS may appear in IT industry in the future. It is good because users can have more choices  to reach their goal.  You can combine tools/ modules and try to pursue your own "one to one engagement".

 

 

Since Facebook appeared in 2004,  SNS and message tools are getting popular and popular, especially in younger generations all over the world.  In principle, communications in SNS and message tools are one to one basis.  Therefore, it is natural that marketing activities by companies are also shifting from mass communication-type marketing to one to one engagement.  Mobile phones will be available at lower cost in emerging markets in the future and more people will be connected to the internet. It means that one to one engagement will be more important than it is now for companies that want to reach customers.

Although there are overlaps between two big IT software companies,  it seems that their partnership is strengthened going forward. I would like to keep watching what is going on between the two companies.  It must be exciting, isn't it?

 

 

Source

1. Salesforce and Microsoft

https://www.salesforce.com/campaigns/microsoft/

2. Twitter of Salesforce

https://twitter.com/salesforce/status/645749326069780480?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet

 

Salesforce, Dreamforce and others are trademarks of salesforce.com, inc. and are used here with permission.

Microsoft, Encarta, MSN, and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I and TOSHI STATS.SDN.BHD. accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

Revolution or evolution? Who wins, Google or Toyota in 2025?

car-407165_640.jpg

Google is a big IT company and Toyota is a giant of automobile industries. They may be categorized in different industries.  However, they must be in tight competition in future because both of them will provide automobiles with artificial intelligence (AI).  Let us consider this competition here.  On 4 Sep, 2015,  Toyota announced that it  collaborates with MIT and Stanford University to develop artificial intelligence to be used in automobiles  and robotics(1).  Toyota thinks drivers can be supported by AI effectively. On the other hand, Google has been developing “self-driving car”, which needs no human  interventions. It seems that Google hired John Krafcik, a car industry veteran who previously led Hyundai’s business in the U.S  to work at Google self-driving car project (2).  It seems that the competition between Google and Toyota has already started.

 

1  Definition of “automobile”

Although both of them use the term of “car”,  each “car” is a little different from each other. According to Toyota, drivers should control their car while AI can support and assist drivers.  On the other hand,  Google promotes self-driving cars, which can drive without human interventions.  The purpose of each “automobile” is the same. It is the transportation from place A to place B.  However, each automobile looks completely different. Toyota AI automobiles might look the same as ordinal automobiles.  They have a steering wheel, accel and brake pedal. On the other hand, Google self-driving cars have no steering wheel, no accel and brake pedal. They may be more compact than ordinal automobiles.

 

2  Self-driving car technologies and human centric technologies

Technologies of each “automobile” are similar.  Both use state of the art technology “Artificial intelligence”.  But the aim of each is a little different.  It means that “Support” vs. “Control”.  I remembered Toyota used “Fun to drive” as a corporate statement in the latter half of 1980s.   Driving is fun because drivers can control cars by themselves. And Toyota uses the terms of ” human centric technologies”.  In my view,  Toyota thinks AI exists to support drivers and it is important to keep a good relationship between human and AI while human should play a major role in driving.  On the other hand,  Google thinking is simple. AI can control automobiles better than human. That is it!

 

3  Revolution or evolution?

The self-driving car is completely new for us in our daily lives. So it sounds revolutionary.  On the other hand, Toyota AI car still needs driver and AI can assist drivers. So it sounds an evolution to me.  Evolution, however, is not always easier than revolution. Because Toyota “human centric technologies” should include “human being” as a major part of the system. According to the video on Toyota websites, Toyota focus on collaboration between human and artificial intelligence. Therefore, human behavior should be analyzed and predicted so that AI knows when and how AI intervenes control of the automobile. If it is not accurate,  this system does not work effectively.  It seems to be more difficult than the system without human intervention is.  AI should learn not only automobile behavior, but drivers’ behavior.  As long as drivers take control automobiles, it is necessary and critically important for automobiles with AI.

 

 

Which automobile do you like better?  The self-driving car is OK for you?  Consumers may have different opinions by country. For example, ASEAN countries, including Malaysia are in the time of motorizations.  Therefore, many consumers want to own and drive their cars by themselves. In addition to that,  the train systems are not so convenient yet in most of the regions so they need their own cars anyway.  On the other hand,  in Japan, consumers are not so enthusiastic in owning cars anymore, especially for younger generations.  For example, in 1980s, Japanese automobile companies produced many sports cars, which were stylish and reasonable for young consumers. They were very popular at that time. Now there are some because sports car is not so popular for younger generations in Japan anymore. In additions to that, in the big cities of Japan such as Tokyo and Osaka,  there are a lot of train networks so there is no need to own cars in daily lives.  Therefore Japanese consumers may be more likely to accept self-driving cars. I am sure each country should consider regulations about automobiles with AI carefully based on the needs and  preferences of the people.

No one knows who wins Google or Toyota In 2025. But I am sure we need a lot of discussions about regulations, insurance, public transportation planning, jobs, and so on. I would like to keep watching it going forward.

 

 

 

Source

1. Toyota Establishes Collaborative Research Centers with MIT and Stanford to Accelerate Artificial Intelligence Research,  website of Totoya motor, 4, September 2015

http://newsroom.toyota.co.jp/en/detail/9233109/

 

2.  Yes, true: I’m joining the Google Self-Driving Car project in late September. 13, September 2015, Twitter of John Krafcik

https://twitter.com/johnkrafcik?lang=en

 

3. Google Self-Driving Car Project

http://www.google.com/selfdrivingcar/

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

What will the food delivery service be in 2025? Let us imagine now!

This week’s topic is about food delivery service in the future.  Syikin,  one of the readers of my articles,  requests me to write this. So I would like to go to Kuala Lumpur in 2025,  10  years later from now.  Let us imagine what happens there!

 

Now we are in 2025. I am working in the office at KLCC.  While everything is changed from 10 years ago, twin tower still remains the same.  At lunch time, I always use the service of food delivery called “something to eat”.   It is a start-up business out there that does research and invents smart food delivery service. I like this service very much.  I use it 3 or 4 times a week because it is convenient and reasonable.  On 11:00AM everyday,  the recommendation of menus for lunch is sent to my mobile phone.  This is not just a picture of dishes. This is 3D virtual reality and can be projected on my desk. These dishes look so real!  It must be good for their sales, too. Their recommendations are completely personalized for me. The service has a historical data about orders from me in the past. So it understands my preference of taste, ingredients, cost sensitivity, perfectly.  I can order dishes by just saying ” I want Japanese potato salad, Chinese fried rice and coke” because computers can understand English conversations just like us. There is no need to use buttons on the screen of a smart phone anymore.  Payment is made by digital currency automatically. Food and dishes are delivered by self-driving cars in the city.  There is less traffic jam recently because traffic can be control by computers so they can deliver efficiently and just in time.  I do not worry about my lunch at all. Just enjoy it!

 

This service is a 24 hour operation so I can order anytime I want. It is very good when I have a night shift work. I am wondering who works at midnight in the kitchen to serve dishes.  I ask public relation personnel for this service about that.  She said “There are automated factories to cook dishes. There is no human operation. Our chef robots are so intelligent that they can cook many dishes.  Each robot has artificial intelligence in it. “.   It sounds like automobile factories in 2015.  In addition to that, many vegetables are also brought up at the factories, too.  So there is no worry about procurements of vegetables even though we have massive natural disasters such as flood, drought and typhoon or cyclone, which cause diminish of supply of vegetables.

 

One of my favorite services is about the nutrition report.  This report is prepared and sent to me at the end of the month.  It tells me what I ate, how many calories there are in my dishes and what kind of nutrition is needed to keep healthy.  It is good to monitor my daily meals so that I can live a healthy life.  Actually, I like sweet things very much so there are sometimes warnings about over-calories. Yeah, I should control it despite its temptation is very strong!

 

I have heard that their predictions of sales of each dish are very accurate because the company developed state of the art statistical models by itself.  It enables them to have less waste because it can control inventories based on the predictions of sales. It is good not only financial conditions of the company, but the environment in society.

 

Recently,  the Japanese government announced that it will introduce basic income to support poor people in 2026  The Japanese economy has been shrinking for a long time and poor people are increasing rapidly.  Some households can not sustain the basic meals every day as they are unemployed.  By introducing a basic income,  they can obtain basic meals everyday.  Food delivery service “something to eat” supports Japanese government to establish meal stations all over Japan.  Because “something to eat” has superior technologies about cooking, serving and predictions of sales. The service can provide basic meals with less cost.  Japanese government expects it works very well.  Especially households with single parent need this support urgently.

 

 

This story is a fiction based on my imagination. So I am not so sure it happens in this way. But  I am sure the more data are obtained, the more personalized and more efficient services are.   Robots with artificial intelligence start working in service industries. In 10 years from now, they must be far more intelligent than they are now.  I would like to go to “something to eat” in 2025!

 

 

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I and TOSHI STATS SDN.BHD. accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

Linkedin bought a predictive marketing company. What does it mean?

I think many people like Linkedin as a platform of professionals and they are interested in what is going on in the company.  Last week, I found that “Today we are pleased to announce that we’ve acquired Fliptop, a leading provider of predictive sales and marketing software.1″  (David Thacker said on the blog on 27 August, 2015 ).  Linekdin bought a leading predictive marketing company.  What does it mean? Let me consider a little.

 

1. What does “Fliptop” do?

It is a marketing software company. On the website, it says  “DRIVE REVENUE FASTER WITH PREDICTIVE MARKETING”, “Increase lead conversion rates and velocity.” and “Identify the companies most likely to buy”.  It was established in 2009 so it is a relatively young company. The company uses technologies called “Machine learning” to identify potential customers with high probability of purchase of products and services.   According to the website of the company, it has an expertise of standard machine learning algorithms, such as logistic regression and decision trees. These methods are used for classifications or predictions.  For example,  the company can identify who is likely to buy the products based on data, including the purchase history of each customer in the past. It hires computer science experts to develop the models for predictions.

 

2. What will Linkedin do with Fliptop?

As you know, Linkedin has a huge customer base so it has a massive amount of data generated by users of Lnkedin everyday.  This data have been accumulated every second. Therefore Lnikedin should have an ability and enhance it to make the most out of the data.  Linkedin should analyze the data and make better business decisions to compete other IT companies in the markets. In order to do that, there are two options, 1. Technologies developed In-house,  2. Purchase of resources outside the company.  Lnkedin took an option of “2” this time. Doug Camplejohn, CEO of Fliptop, said “We will continue to support our customers with existing contracts for some period of time, but have decided not to take on any new ones. We will also be reaching out to our customers shortly to discuss winding down their existing relationship with Fliptop.”.  Therefore Fliptop will not be independent as a service provider and will be integrated into the functions of Linkedin. It seems that knowledge and expertise of Fliptop are seamlessly integrated into Linkedin in future.  I am not so sure what current users of Fliptop should do as long as I know now.

 

3.  Data is “King”

This kind of purchases has been seen in IT industry recently. Google bought “DNN research” in 2013 and “DeepMind” in 2014. Microsoft also bought “Revolution Analytics” in 2015.  These small or medium size companies have expertise in machine learning and data analysis.  When they try to expand their businesses, they need massive data to be analyzed. However, they are not owners of a massive amount of data. Owners of a massive amount of data are usually big IT companies, such as Google and Facebook.  It is sometimes difficult for relatively small companies to obtain a massive amount of data, including  customer data.  On the other hand, big IT companies, including Linkedin, are usually owners of huge customer data. In addition to that, big IT companies now  enhance resources and expertise to analyze data as well. Once they have both of them, new services can be created and offered in a shorter period. The more people use these services, the more accurate and effective they can be.  Therefore, it sounds logical when big IT companies acquire small companies with expertise in data analysis and machine learning. Big IT companies definitely need their expertise in data analysis and machine learning.

 

 

From the standpoint of consumers, it is good because they can enjoy many services offered by big IT companies with lower costs. But from the standpoint of companies, competitions are getting tougher as this occurs not only in IT industries but many other industries. Now Linkedin seems to be ready for this competition, which comes in the future.

Machine learning is sometimes considered as engines and data are considered as fuel.  When they are combined in one place, new knowledge and insights may be found and new products and services may be created.  It accelerates changes of the landscape of the industries. Mobile, cloud, big data, IOT and artificial intelligence will contribute to this change a lot. It must be exciting to see what happens next in the future.

 

 

 

Source

1. Accelerating Our Sales Solutions Efforts Through Fliptop Acquisition, David Thacker, August 27, 2015 

http://sales.linkedin.com/blog/accelerating-our-sales-solutions-efforts-through-fliptop-acquisition/

2.A New Chapter,  Doug Camplejohn, August 27, 2015

http://blog.fliptop.com/blog/2015/08/27/a-new-chapter/

 

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user.

How can we create good movies based on big data?

Last Sunday, my son came to Kuala Lumpur as he has summer vacation now.  So  I brought him to the movie theater.  I chose “Mission: Impossible – Rogue Nation” to entertain him.  In the movie, Tom cruise is very active.  I cannot believe he is older than I am!  My son and I could enjoy the movie very much, as his action is amazing. Then I am wondering how we can create good movies.  Every year, many movies are created, but few of them can stay in people’s mind in the long term.  Let me consider it here for a while.

 

1. How can we define what good movies are?

There are many measures to evaluate movies.  Critics can assess the quality of movies. But I would like to make it simple.  Number of customers who watch the movie or the amount of sales revenue such as “Box office“of the movie can be used as a measure of “good movie” as it is easy to collect and measure. So the more people watch the movie, the better it is according to our definition of “good movies”.

 

2. Let us consider something related to the number of customers or sales revenue.

A lot of things relate to them.  Like Mission:Impossible,  actors and actresses are very important.  The director is also important.  In addition to that,  where is it created?  Is it an action movie or a love story or a thriller?,  and so on. They may be closely related to the number of customers or sales revenue of the movie.  So the data about something related to the number of customers or sales revenue in the past should be collected.

 

3. How can we obtain predictions of the number of customers or sales revenue of the  unseen movie in advance?

Could you remember the last week’s letter about “Target” and “Features”?  “Target” is something that we want to predict and “Features” are something that are related to “Target”.  Predictions of “Target” can be obtained by inputting “features” into “Statistical model”.  I would like to call this unit “module”.   I summarize it as follows.

According to our definition of “good movies”,  Target is the number of customers or sales revenue of the movie. Features are actors and actress, category of the story, locations where the movie was taken, and so on. So these features are input to statistical models to obtain predictions of target for unseen movies.  Based on this analysis,  we could predict the sales of movies before they are seen in theaters. It means that good movies could be created based on this prediction. When this prediction is accurate,  film production companies might increase sales revenuebecause they can create good movies based on predictions of Targets. But in reality, we should prepare a lot of data to predict them accurately.  In additions to that,  customer preference might be changed suddenly, however, it is very difficult to update the statistical models in advance to follow such changes. Therefore, there is a risk where statistical models can not follow circumstance changes in a timely manner.

 

 

It should be noted that more features will be available as computers will understand videos or movies. Now the technology 1is in progress.  It will enable computers to turn videos into texts. For example, when there is a scene where the swan is on the lake, computers understand the video and make sentences that explain the scene automatically.  It means that whole part of the movie can be transformed into texts without human intervention. So movies will be analyzed based on their stories. More features can be identified in the results of this analysis. When new kinds of data are available to us, it may enable us to obtain more features and improve accuracy of predictions.  Would you likely to make your own movie in future?

 

Source

1. A picture is worth a thousand (coherent) words:building a natural description of images, 17 Nov 2014,  Google Research

http://googleresearch.blogspot.co.uk/2014/11/a-picture-is-worth-thousand-coherent.html

 

 

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user

"Prediction" is very important in analyzing big data of the business

It is a good timing to reconsider “Big data and digital economy” because this name of group on Linledin has four-month-history and more than 100 participants now. I would like to appreciate the cooperation of all of you.

In the beginning of 2000s, I worked in the risk management dept in the Japanese consumer finance company.   There is a credit risk model which can predict who is likely to be in a default in the company. I learned it more details and understood how it worked so accurately. I found that if I collect a lot of data about customers, I could obtain accurate predictions for events of defaults in terms of each customer.

Now in 2015,  I researched many algorithms and statistical models including the state of art “deep learning”.   While there are many usages and objectives in using such models,  in my view,  the most important thing for business persons is “prediction” just like my experience in consumer finance company because they should make good business decisions to compete in markets.

If you are in health care industry,  you may be interested in predictions about who is likely to be cured. If you are in sales, you may be interested in predictions about who is likely to come to the shop and buy the products. If you are in marketing,  you may be interested in who is likely to click the advertisement on the web.  Whatever you do,  predictions are very important for your businesses because it enables us to take the right actions.  Let me explain key points about predictions.

 

Target

What are your interests to predict?    Revenue of your business?  Number of customers?    Satisfaction rate based on client feedback?  Price of wine near futures? You can mention anything you want.  We call it “Target”.  So firstly, “Target” should be defined in predictions so that you can make right business decisions.

 

Features

Secondly,  let us find something related to your target.  For example,   If you are a sales person and interested in who is likely to buy the products,  features are “attributes of each customer such as age, sex, occupation” , “behavior of each customer such as how many times he/she come to the shop per month and when he/she bought the products last time”,  “What did he/she click in the web shop”  and so on.  Based on the prediction, you can send coupons or tickets to “highly likely to buy”customers in order to increase your sales.  If you are interested in the price of wine,  features may be temperature,  amount of rain and locations of farms,  and so on.  If you can predict the price of wine,  you might make  good investments of wine.  These are just simple examples. In reality,  a number of features may be 100,  1000  or more.  It depends on whole data you have.  Usually the more data you have, the more accurate your predictions are.  This is why data is very important to obtain predictions.

 

Evaluation of predictions

Finally by inputting features into statistical models,  predictions of the target can be obtained. Therefore, you can predict who is likely to buy the products when you think of marketing strategies.  This is good for your business as marketing strategies can be more effective.  Unfortunately customer preferences may be changed in the long run.  When situations and environments such as customer preferences are changed,  predictions may not be accurate anymore.  So it is important to evaluate predictions and update statistical models periodically.  No model can work accurately forever.

 

Once you can obtain the prediction,  you can implement processes of the predictions as a daily activity, rather than one-off analysis. It means that data driven decisions are made on a daily basis.  It is one of the biggest aspects of “digital economy”.  From retail shops to health care and financial industry,  predictions are already used in many fields.  The methods of predictions are sometimes considered as “black-box”.  But I do not think It is good to use predictions without understanding the methods behind predictions. I would like to explain them in my weekly letter in future.  Hope you enjoy it!

 

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user. 

Could you win the game "Go" against computers? They are smarter now!

There are many board games all over the world. I think you enjoy Chess, Othello, Backgammon, Go, and so on.  Go is the game where two players put the white stone and black stone in turn and decide the winner by comparing the areas owned by each player. You might see Go before like the image above.   I learned how to play Go when I was in the elementary school and enjoy it since then.

In most of the board games,  even top professional human players feel difficulties to beat the artificial intelligence (AI) players.  One of the most famous stories about competitions between human and computers is that "Deep Blue vs. Garry Kasparov " ,  six-game chess matches between chess champion Garry Kasparov and an IBM supercomputer called Deep Blue.  In 1997  Deep Blue defeated  Garry Kasparov. It was the first win by computers against world chess champions under the tournament regulations1.

However Go is still dominated by human players.  Top professional Go players are still stronger than AI players while they are getting better by improving algorithms. Crazy Stone is one of the strongest Go playing engines, developed by Rémi Coulom. On 21 March, 2014, at the second annual Densei-sen competition, Crazy Stone defeated Norimoto Yoda, Japanese professional 9-dan, in a 19x19 game with four handicap stones by a margin of 2.5 points 2. On 17 March 2015,  Chikun Cho (The 25th Hon'inbo) defeated Crazy Stone in a 19x19 game with three handicap stones by a margin of 0.5 points by resignation (185 moves)3.   Human player won against AI player in the game.  But handicap is smaller from four stones in 2014 to three stones in 2015.  I am not sure human players continue to win the competition in 2016.

For the AI players like Crazy Stone, the secret is the technology called "Reinforcement learning" which is used for selecting actions to maximize future reward.  So this can be used to support decision making, such as investment management,  helicopter control and advertizing optimizations.  Let me look at the details.

 

 

1. Reinforcement learning can handle delayed rewards

Unlike quiz shows,  it takes time to realize whether each action is good or bad for board games.  For example, a board of Go has a grid of 19 lines by 19 lines. So at the beginning of the game, it is difficult to know if each action is good or bad as we have a long way to the end of the game. In other words, A reward by each action is not provided immediately after it is taken. Reinforcement learning has  a mechanism to handle such cases.

 

2. Reinforcement learning can calculate optimal sequential actions

In Reinforcement learning, agents play a major role.  Agents can take actions based on their observations and  strategy.  Actions can be formed as "path", not just one-off action. This is similar to our decision making process. Therefore, Reinforcement learning can support human decision making.  Actions are usually considered to have no impact against the environment.

 

3. Reinforcement learning is flexible enough to use many methods of searching

This is practically important.  Like Go, some problems have a huge space to search for optimal actions. Therefore, we need to try several methods to do that. Reinforcement learning is flexible to try these search methods.

If you would like to study it more details,  I recommend lectures by David Silver, Google DeepMind London, Royal Society University Research Fellow, University College London.

 

 

In future, a lot of devices will have sensors in them and be connected to the internet. Each device will send information, such as locations, temperatures, weather periodically. Therefore the massive amount of time series data is generated, collected automatically through the internet.  Based on these data,  we need sequential actions to maximize rewards.   If we have data from engines in automobiles,  we should know when a minor repair is needed and when an overhaul is needed to make engines work for a longer period..  If we have data from customers, we should know when notifications of sales should be sent to maximize the amount of sales in the long run.  Reinforcement learning might be used to support this kind of business decisions.

I would like to develop my own AI Go players better than I am.  It must be fun to have games with them!  Would you like to try it?

 

 

 Source

 1. Deep Blue versus Garry Kasparov

https://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov

 2. Denseisen (Japanese only)

http://entcog.c.ooco.jp/entcog/densei/past.html

 3. 2nd game in the 3rd Denseisen 

http://entcog.c.ooco.jp/entcog/densei/densei3/2nd_game.html

 

 

 

Note: Toshifumi Kuga’s opinions and analyses are personal views and are intended to be for informational purposes and general interest only and should not be construed as individual investment advice or solicitation to buy, sell or hold any security or to adopt any investment strategy.  The information in this article is rendered as at publication date and may change without notice and it is not intended as a complete analysis of every material fact regarding any country, region market or investment.

Data from third-party sources may have been used in the preparation of this material and I, Author of the article has not independently verified, validated such data. I accept no liability whatsoever for any loss arising from the use of this information and relies upon the comments, opinions and analyses in the material is at the sole discretion of the user.