European GDPR and its impact on Australian organisations. We give you the low-down from an analytic tool perspective.

What is the European GDPR and how will it impact Australian organisations?  We give you the low-down from an analytic tool perspective.

GDPR (General Data Protection Rules) is the European privacy and data protection law that comes into effect on the 25th of May 2018.  This surely doesn’t affect Australian companies, right? Wrong!

The thing is, whilst the new regulation governs data protection and privacy for all EU citizens it also addresses personal data outside of the EU. The impact will be far-reaching, including Australian businesses, as all businesses concerned with the gathering and analysis of consumer data could be affected.

What the law says

According to the Office of the Australian Information Commissioner (OAIC), Australian businesses of any size may need to comply. In addition, all Australian businesses must comply with the Australian Privacy Act 1988.

Are these two laws complimentary? Some of the common requirements that businesses must adhere to include:

  • Implementation of privacy by design approach to compliance
  • An ability to demonstrate compliance with privacy principles and obligations
  • Adoption of transparent information handling practices
  • Appropriate notification in case of any data breach
  • Conduction of Privacy impact assessments

But some GDPR requirements are not part of the Australian Privacy Act, such as the “right to be forgotten”.

What now?

We would suggest that Australian businesses firstly establish whether they need to comply with GDPR.  If they do, then they should take prompt steps to ensure their data practices comply. Businesses should already comply with the Australian Privacy Act, but also consider rolling out additional measures required under GDPR which are not inconsistent with the Privacy Act.

Who is affected

In a nutshell, the GDPR applies to any data processing activities undertaken by an Australian business of any size that:

  • Has a presence in the EU
  • Has a website/s that targets EU customers or mentions customers or users in the EU
  • Tracks individuals in the EU to analyse (for example to predict personal preferences, behaviours and attitudes)

Refer to the following link for more information: https://www.oaic.gov.au/media-and-speeches/news/general-data-protection-regulation-guidance-for-australian-businesses

Do analytic tools comply?

Once a need for your organisation to comply has been established, it is worth ascertaining whether the actual tools you are using for analytics comply; specifically regarding the last bullet point above (tracking and analysing individuals).

In the next section of this article we look at two common players in the analytics space; Power BI and Qlik, through the lens of GDPR (and by default the Australian Privacy Act).

The scope of GDPR is intended to apply to the processing of personal data irrespective of the technology used. Because Power BI and Qlik may be used to process personal data, there are certain requirements within the GDPR that compel users of these technologies to pay close attention:

  • Article 7 states that consent must be demonstrable and “freely given” if the basis for data processing is consent.  The data subject must also have the right to withdraw consent at any time
  • Articles 15 to 17 covers the rights to access, rectification, and erasure. This means that mechanisms must allow data subjects to request access to their personal data and receive information on the processing of that data. They must be able to rectify personal data if it is incorrect. Data subject must also be able to request the erasure of their personal data (i.e. the “right to be forgotten”)
  • Articles 24 to 30 require maintenance of audit trails and documentary evidence to demonstrate accountability and compliance with the GDPR
  • Article 25 requires businesses to implement the necessary privacy controls, safeguards, and data protection principles so that privacy is by design
  • Articles 25, 29 and 32 require strict data security access control to personal data through for example role-based access and segregation of duties

Microsoft Power BI

Power BI can be viewed through the lens of GDPR (and the Australian Privacy Act for that matter) via four pillars in the Microsoft Trust Centre. With specific reference to GDPR, Microsoft states, “We’ve spent a lot of time with GDPR and like to think we’ve been thoughtful about its intent and meaning”.  Microsoft released a whitepaper to provide the reader with some basic understanding of the GDPR and how it relates to Power BI. But meeting GDPR compliance will likely include a variety of different tools, approaches, and requirements.

Security

Power BI is built using the “Security Development Lifecycle”, Through Azure Active Directory Power BI is protected from unauthorised access by simplifying the management of users and groups, which enables you to assign and revoke privileges easily.

Privacy

The Microsoft Trust Centre clearly states that “you are the owner of your data” and it is not used for mining for advertising.  http://servicetrust.microsoft.com/ViewPage/TrustDocuments?command=Download&downloadType=Document&downloadId=5bd4c466-277b-4726-b9e0-f816ac12872d&docTab=6d000410-c9e9-11e7-9a91-892aae8839ad_FAQ_and_White_Papers

From the Power BI white paper, “We use your data only for purposes that are consistent with providing the services to which you subscribe. If a government approaches us for access to your data, we redirect the inquiry to you, the customer, whenever possible. We have challenged, and will challenge in court, any invalid legal demand that prohibits disclosure of a government request for customer data.” https://powerbi.microsoft.com/en-us/blog/power-bi-gdpr-whitepaper-is-now-available/  

Compliance

Microsoft complies with leading data protection and privacy laws applicable to Cloud services, and this is verified by third parties.

Transparency

Microsoft provides clear explanations on:

  • location of stored data
  • the security of data
  • who can access it and under what circumstances

Qlik

The BI vendor, Qlik, released a statement that declares “With more stringent rules and significant penalties, GDPR compels businesses to use trusted vendors. Qlik is committed to our compliance responsibilities – within our organization and in delivering products and services that empower our customers and partners in their compliance efforts.” – https://www.qlik.com/us/gdpr

Qlik released an FAQ document as a GDPR compliant vendor stating that they have various measures in place to protect personal data and comply with data protection/privacy laws, including GDPR:

  • Legal measures to ensure the lawful transfer
  • Records of data processing activities (Article 30)
  • Ensuring Privacy-By-Design and Privacy-By-Default
  • Data retention and access rules
  • Data protection training and policies

For more information, please view the links below:

https://www.qlik.com/us/-/media/files/resource-library/global-us/direct/datasheets/ds-gdpr-qlik-organization-and-services-en.pdf?la=en

Conclusion

The two vendors discussed are clear in their commitment to ensuring their security arrangements can comply with GDPR. This does not mean that other major players (Tableau, Google, etc.) do not have the same initiatives in flight, we have only focused on Microsoft and Qlik.

Whilst there is no ‘magic button’ available to ensure all regulations are miraculously met, it is possible regardless of vendor:

  • To ensure security policies can meet GDPR compliance
  • To design with privacy in mind.  Even though platforms may meet “privacy is by design”, your specific solution must still be proactively designed.  You cannot simply rely on the vendor
  • To conduct an appropriate solution audit with aligned to GDPR (or Australian Privacy Act) as a good final step

GDPR can indeed be a tricky landscape to navigate – if in doubt, check it out.

We can certainly assist in guiding you through the process from an Data and Analytics perspective.

A Power BI Cheat Sheet – demystifying its concepts, variants and licencing

Power BI has truly evolved over the past few years.  From an add-on in Excel to a true organisation wide BI platform, capable of scaling to meet the demands of large organisations; both in terms of data volumes and the number of users. Power BI now has multiple flavors and a much more complicated licencing model. So, in this article, we demystify this complexity by describing each flavor of Power BI and their associated pricing. We summaries it all at the end with some scenarios and in a single cheat sheet for you to use.

Desktop, Cloud, On-premise, Pro, Premium, Embedded – what does all of this mean?

I thought it best to separate the “why” (i.e. why do you use Power BI – Development or Consumption), the “what” (i.e. what can you do given your licence variant), and the “how much” (i.e. how much is it going to cost you) as combining these concepts often leads to confusion as there isn’t necessarily an easy map of why what and how much.

Let’s first look at the “why”

“Why” deals with the workload performed with Power BI based on its deployment – I.e. why do you use Power BI? Is it for Development or for Consumption. This is very much related to the deployment platform (i.e. Desktop, Cloud, On-Premise or Embedded).

The term “consumption” for the purpose of this article could range from a narrow meaning (I.e. the consumption of Power BI content only) to a broad meaning (i.e. consumption of-, collaboration over-, and management of Power BI content – I refer to this as “self-serve creators”).

Why – workload/ deployment matrix

Now let’s overlay the “why” with “what”

In the table above, I not only dealt with the “why”, but I also introduced the variants of Power BI; namely Desktop, Free, Pro, On-Premise and Embedded. Variants are related to the licence under which the user operates and it determines what a user can do.

Confused? Stay with me…all will become clearer.

What – deployment/ licence variant matrix

Lastly let’s look at the “how much”

The Power BI journey (mostly) starts with development in Desktop, then proceeds to a deployed environment where it is consumed (with or without self-serve). Let’s close the loop on understanding the flavours of Power BI by looking at what this means from a licencing cost perspective.

Disclaimer: The pricing supplied in the following table is based on US-, Australian-, New Zealand- and Hong Kong Dollars. These $ values are by no means quotes but merely taken from the various calculators and pricing references supplied by Microsoft as at the date of first publication of this article.

How much – licence variant/ cost matrix

https://www.microsoft.com/en-Us/sql-server/sql-server-2017-pricing

https://powerbi.microsoft.com/en-us/calculator/

https://azure.microsoft.com/en-us/pricing/calculator/

**Other ways to embed Power BI content are via Rest API’s (authenticated), SharePoint online (via Pro licencing) and Publish to Web (unauthenticated), but that is a level of detail for another day. For the purpose of this article, we focus on Power BI Embedded as the only embedded option.

Pro is pervasive

Even if you deploy to the Cloud and intend to make content available to pure consumers of the content only (non-self-serve users), whether it be in PowerBi.com or as embedded visuals, you will still need at least one Pro licence to manage your content. The more visual content creators (self-server creators) you have, the more Pro licences you will need. But, it is worth considering the mix between Pro and Premium licences, as both Pro and Premium users can consume shared content, but only Pro users can create shared content (via self-service), so the mix must be determined by a cost vs capacity ratio (as discussed below).

A little bit more about Premium

Premium allows users to consume shared content only. It does not allow for any self-service capabilities. Premium licences are not per user, but instead, based according to planned capacity, so you pay for a dedicated node to serve your users. Consider Premium licencing for organisations with large numbers of consumers (non-self-serve) that also require the dedicated computer to handle capacity. The organisation would still require one or more Pro licences for content management and any self-serve workload.

Premium licencing is scaled as Premium 1, 2 or 3 dependant on the number of users and required capacity. You can scale up your capacity by adding more nodes as P1, P2 or P3, or scale up from P1 to P2, and from P2 to P3.

Premium capacity levels

The mix between Pro and Premium

Given that Pro users can do more than Premium users, and given that you will need to buy one or more Pro licences anyway, why would you not only use Pro rather than Premium? There are two reasons:

  • There is a tipping point where Pro becomes more expensive compared to Premium, and
  • With Pro licences you use a shared pool of Azure resources, so is not as performant as Premium which uses dedicated resources, so there is a second tipping point where your capacity requirements won’t be sufficiently served by Pro.

The diagram below shows the user and capacity tipping points (discussed further in scenario 1 below):

Capacity planning Premium 1 vs Pro: Users/ Cost/ Capacity

Put this all together

Right, you now understand the “why”, “what” and “how much” – let’s put it all together through examples (I will use Australian $ only for illustrative purposes). Please note that there are various ways to achieve the scenarios below and this is not a comprehensive discussion of all the options.

Scenario 1

A large organisation has 10 Power BI Developers; their Power BI rollout planning suggest that they will grow to 50 self-service creators and 1450 additional high activity consumers in 12 months. And that they will grow to 125 self-serve creators and 5000 high activity consumers in 48 months:

Initially, they will require

10 x Power BI Desktop licences = $0 x 10 = $0

500 x Power BI Pro licences to cover both self-serve users and consumers = $12.70 x 500 = $6,350

Total – A$6,350.00pm

Once they exceed 500 they can revert to

50 x Power BI Pro licences to cover self-serve users = $12.70 x 50 = $635

1 x P1 node to cover the next tranche of high activity consumers = $6,350

Total – A$6,985.00pm

Thereafter

Add Power BI Pro licences as required up to their planned 125 = $12.70 x 125 = $1,588

Add 1 additional P1 node at 1,450 users, and again at 2,900 users, and again at 4,250 users = $25,400 for 4 x P1 nodes

Total after 4 years at 5000 high activity consumers and 125 self-serve creators – A$26,988.00pm

Scenario 2

A small organisation with 1 Power BI developer, 5 additional self-service creators and 10 additional consumers of visual content, with no custom applications/ websites.

1 x Free version of Power BI Desktop: 1 x $0

15 x Pro licences as both visual creators and mere consumers will take part in shared content: 15 x $12.70

Total – A$190.50pm

Scenario 3

A small ISV organisation with 3 Power BI developers want to embed Power BI content in an application that they sell. The application must be up 24 x 7 and do not require a very high volume of concurrent users, but licencing cannot be on a per-user basis.

3 x Free version of Power BI Desktop: 3 x $0

1 x Pro licences acting as the mater of the Shared content: 1 x $12.70

A1 Node pricing: 1 x $937

Total – A$950.00pm

Scenario 4

A medium sized organisation with 5 Power BI developers want to embed Power BI content in an internal portal such as SharePoint which is used by potentially 250 users. They also have 10 self-service creators and 25 consumers of Power BI content through the Power BI portal.

5 x Free version of Power BI Desktop: 3 x $0

26 x Pro licences acting as 1 mater of the Shared content and 25 consumers: 26 x $330.20

A1 Node pricing: 1 x $937

Total – A$1,267.20pm

Power BI – licence variant, workload, deployment & cost cheat sheet

Any process is shown in Australian $

Disclaimer: The pricing supplied in the following table are by no means quotes, but merely taken from the various calculators and pricing references supplied by Microsoft as at the date of first publication of this article.

Licence variant, workload, deployment & cost cheat sheet

Networks Asset Data Mart – our Energy Infrastructure Provider case study

networks asset

Exposé designed and developed a solution that saw an increasingly temperamental Networks Asset Analytical solution move to the Exposé developed Enterprise Analytics Platform.

The solution now:

• Allows staff to focus on business-critical tasks by utilising the data created by the system.
• Reduces support costs due to the improved system stability.
• Utilises the IT resources for other projects that improve business productivity.

exposé case study – Energy Infrastructure Provider – Networks Asset Data Mart

See another case study here

An Internet of Value – Blockchain, beyond the hype and why CxO’s must take note

A Blockchain, in its simplest form, is a distributed database system where there is no one master (primary) database, but many databases that are all considered primary. All parties participate in populating entries into the respective databases and receive the entries of the other participants.

But how does this apply to your business, and is this profoundly going to change how the world works? Let’s look at an analogy: Imagine I create a song and generate a file of my recording in mp3 format on a USB stick. I can give two of my friends a copy of this; they can do the same, and so on. With thousands of eventual copies going around, it will be impossible to establish which was the real version I own and which I ideally wanted to use in exchange for royalties. By the way, if I ever had to create a song and recorded it, I doubt very much that I would garner thousands of fans. I am just not David Grohl 😊

This is where Blockchain comes in. It is a shared ledger that is used to record any transaction and track the movement of any asset whether tangible, intangible or digital (such as my mp3). It is immutable, so participants cannot tamper with entries, and it is distributed, so all participants share and validate all the entries.

Blockchain will allow “my fans” 😊 to enter into a contract with me directly. As they stream the song, payment goes directly from their wallet into mine. The information about what was listened to and what I was paid, is verified by all the databases in the network and cannot be changed. There are no middlemen (like a central streaming service, or a record label), so the contract (a digital smart contract) is between those that listen to my song and me directly.

It is at this point important to mention that Blockchain is not Bitcoin, or any other cryptocurrency, although it did start life as the technology that underpins cryptocurrencies. This article, the first in a series of three articles, looks beyond its use in cryptocurrencies, and rather highlights use cases to show CxO’s why it is so important to take note of Blockchain and to start controlled proof of concepts (POC’s) and research and development (R&D) in this technology now. We look at some examples across a wide range of industries and use a Courier based use case to delve deeper into what Blockchain could mean for organisations using the Internet of Things (IoT).

Sport

Dope testing and cheating have been quite topical lately with large portions of the Russian contingent banned from the Rio Olympics in 2016, and again from the Winter Games in South Korea in 2018 for systemic manipulation of tests. Blockchain will make the test results immutable and open the results up to all that participate in the data cycle. Even if the athlete changes sports, that data will be available to participating sporting organisation. http://www.vocaleurope.eu/how-technology-can-transform-the-sports-domain-blockchain-2-0-will-boost-anti-doping-fight-sports-investments-and-e-sports/

Health

Some countries are planning health data exchanges with the aim of addressing a lack of transparency and improving trust in patient privacy as well as fostering better collaboration. Blockchain will provide practitioners and providers with better access to health, patient and research information. Adoption of Blockchain will lead to closer collaboration and better treatment and therapies, sooner.

Blockchain in healthcare is real and imminent. This study from IBM shows how pervasive Blockchain is expected to become with 16% of 200 responding health executives aiming to implement a Blockchain solution shortly. https://www.ibm.com/blogs/think/2017/02/Blockchain-healthcare/

Banking

 Australia’s Commonwealth Bank collaborated with Brighann Cotton and Wells Fargo to undertake the world’s first global trade transaction on Blockchain between independent banks – an unbroken digital thread that ran between a product origin and its final destination, capturing efficiencies by digitising the process and automating actions based on data. https://www.commbank.com.au/guidance/business/why-blockchain-could-revolutionise-the-shipping-industry-201708.html

CommBank is taking this a few steps further with an appointed head of Blockchain and a whopping 25 proof of concepts over the past five years, including the ability to peer-to-peer transfer of funds offshore within minutes rather than days, and the issuing of smart contracts. http://www.innovationaus.com/2017/12/CBA-outlines-a-blockchain-future

Insurance

Customers and insurers will be able to manage claims better, transparently and securely. Claim records, which are tamper proof once written to the chain, will streamline the claim process and minimise claimant fraud such as multiple claims for the same incident.

With Smart Contracts, payments can be triggered as soon as certain minimum conditions are met. There are also many smart contract rules that could ascertain when a claim is also fraudulent automatically denying the claim. https://www2.deloitte.com/content/dam/Deloitte/ch/Documents/innovation/ch-en-innovation-deloitte-Blockchain-app-in-insurance.pdf

Courier Delivery

Couriers deliver millions of items each day, very often crossing vast geographical distances and across multiple sovereign boundaries with unique laws and processes.

These businesses, who often make heavy use of IoT devices, will benefit hugely from Blockchain to improve the ability to track every aspect of a package delivery cycle and minimise fraud.

There were 20 billion connected IoT devices in 2017 and projected to grow to 75 billion by 2025. https://www.statista.com/statistics/471264/iot-number-of-connected-devices-worldwide/

The current centralised approach for insertion and storage of IoT data (see the image below) simply won’t be able to cope with volume demands and transactional contracts will have to rely on multiple 3rd parties. Also Managing data security can be very complex because data will flow across many administrative boundaries with different policies and intents.

In contrast, the Blockchain decentralised peer-to-peer approach for insertion and storage of IoT data eliminates issues with volume demand, (the data is stored across a potentially unlimited number of databases). There is no single point of failure that can bring the whole IoT network to a halt (computation and storage is shared and there is no one primary). It supports tamper-proofing (all participating databases validate a transaction, which is then shared and becomes immutable), which means increased security from rogue participants such as IoT device spoofers and impersonators (Spoofing can occur when security is breached through a lowly secured device on a shared IoT network. If the lowly/ unsecured device is hackable, then the whole network is compromised as it will believe that the hacker is encrypted as the intruder is on it through the easily hacked device).

Delving deeper into our Courier Delivery use case – Blockchain and IoT, creating an Internet of Value

In a courier parcel delivery ecosystems, the movement of parcels is tracked every step of the delivery process via IoT devices that reads a barcode, or another form of identification that can be picked up by the sensor. From the original warehouse to a vehicle, a plane, another warehouse, and finally your home.

By using Blockchain, each sensor participates in the chain and records “possession” of the delivery item (and so also the location). Each time it is read by a new sensor, the new location is broadcast to, inserted, then shared and agreed on by the remaining participants on the Blockchain. Every block is subsequently a transaction that is unchangeable once inserted into the blockchain.

Each Blockchain entry (i.e. the barcode, the location of the package and a date-time stamp) is encrypted into the Blockchain. The “possession” steps are tracked no matter who is involved in the package delivery process (from the origin which could be the delivery at an Aus Post outlet, to an Aus Post vehicle to the airport, to QANTAS en route to the US, to a DHL distribution centre in a US airport, and finally to a DHL delivery vehicle en route the destination address). This enhances trust in the system as there is no need to adhere and interface with a single primary system, and package tracking is put on steroids. If you have ever sent anything abroad, you would know that granular tracking effectively ends at the border. This won’t be the case with Blockchain. https://www.draglet.com/Blockchain-applications/smart-contracts/use-cases

Conclusion

It must be noted that Blockchain technology has not been around for very long and is rapidly evolving. Widespread commercialisation beyond cryptocurrencies is still in its infancy. But all indications are that it will be a hugely disruptive technology.

The many examples of important players taking this technology seriously move Blockchain beyond hype:

CxO’s may ask, why to invest in something they cannot yet fully understand, but this was probably a very similar question asked in the 90’s about the internet. The learning curve will no doubt be steep, but that makes investing in targeted R&D and POC’s early all the more important so that they do not get caught off guard once commercialisation starts increasing.

In the next article, Blockchain, lifting the lid on confusing concepts, we will delve a little bit deeper and describe the concepts in more depth.

Internet of Mice

Advanced analytics

The Internet of Mice – Our IoT and Advanced Analytics Solution

Understanding how animals involved in research move and eliminating as much human handling as possible makes for a much more humane environment for the animals. The outcome is more accurate results for the researchers. See how our IoT and Advanced Analytics solution developed for our customer strives towards a humane research environment and delivers more intelligent insights to researchers.

See more about IoT

Our YouTube channel

Our youtube Channel

We have a growing list of videos on our YouTube channel where you can find some selected case studies, test drives and solutions. Get an inside look at the world of Smart Analytics.

Topics include:  Advanced Analytics, Cognitive Intelligence, Artificial Intelligence, Augmented- and Virtual Reality, IOT and Business Intelligence

Feel free to subscribe as we are constantly adding new videos.

Our YouTube channel

 

Chatbots – how the Azure bot framework is changing the AI game

What are Chatbots?

Communication underpins intelligence. And language underpins communication. But language is complex and must be understood through the prism of intent and understanding. For example:

Take the term, “thong” – in Australian slang this means flip-flops, a meaning lost on someone not familiar with Australian slang, as it means underwear in most other countries.

This is where bots, specifically chatbots come into play. They allow users to interact with computer systems through natural language, and they facilitate the learning and training of, amongst others, language, intention and understanding through machine learning and cognitive APIs.

It is important for the chatbot to be able to leverage trained understanding of human language so that it knows how to respond to the user request, and what to do next. And so, when “John” (who you will meet below) interacts with the computer with the question “do you sell thongs?” the computer understands what it means within the correct context.

Sounds cool, but complicated? Things have become much easier

Five years ago, embarking on a project to build an intelligent chatbot would have been an exercise involving an array of specialists assisting in the interpretation of natural language processing.  It wasn’t something that was affordable for companies other than those in the Fortune 500.

How times have changed – with the development of natural language processing toolkits and bot building frameworks such as wit.ai and api.ai. these tools have allowed web application/lambda developers to have the means to create intelligent yet simple chatbots without the requirement of a natural language processing specialist.

There are different options available to build a chatbot, but in this article, we investigate the Microsoft bot framework and introduce our own EVA (the Exposé Virtual Agent) – a chatbot built within the Microsoft bot framework. But first, let’s have a quick look at why businesses should care (i.e. what are the business benefits)?

Why should businesses care?

It’s mostly about your customer experience!

We have all dealt with customer call centres. The experience can be slow and painful. This is mainly due to the human staff member on the other side of the call having to deal with multiple CRM and other systems to find the appropriate answers and next actions.

Chatbots are different. Providing they can have a conversation with the customer, they are not limited by technology as they have the ability to dig through huge amounts of information to pick out the best “nugget” for a customer. They can then troubleshoot and find a solution or even recommend or initiate the next course of action.

Let’s look at how this can be achieved with the Microsoft Bot Framework.

What is the Microsoft bot framework?

The Microsoft bot framework is a platform for building, connecting, testing and deploying intelligent and powerful bots.  The bot framework works by providing a tool that allows you to bring together all the Microsoft bot related technologies together; easily and efficiently. The core foundation of this framework is the Azure Bot Service.

The Azure Bot Service manages the desired interaction points, natural language processing tools and data sources. This means that all of the interactions go through the bot service before they make use of any natural language or cognitive toolkits, while also using these interactions to utilise information for a variety of data sources; for example Azure SQL Database.

In figure 1, “John” interacts with the Bot Service via a channel (that thing they use to communicate with the Computer in Natural Language). Many readers will already have used Skype and Slack to interact with other humans. They can now use this to interact with Computers too.

Bot Interaction
Figure 1

John is essentially asking about Thongs, its availability and ends up with all the information he needs to buy the product. The Bot framework interacts with the broader Cognitive Services APIs (in this example Language Understanding and Knowledge Base) and various external sources of information, whilst Machine Learning continually learns from the conversation.

Let’s look at a local government example:

A council ratepayer interacts with the council’s bot via the council website and asks for information on the rubbish collection. At this point, the bot will simply refer to a particular knowledge base, and in addition other sources of information such as the website, an intranet site or a database.  The bot’s response is at this stage informative. A response could, for example, be, “Rubbish is collected each week in Parkside on Friday mornings between 530am and 9am. General waste must go in the red bin and is collected each week. Recyclables in the Yellow bin and Garden Waste in the Green bin is alternated each week”.

The user realizes he has no Green bin and so asks the bot where one can obtain a Green bin.

The bot now uses Language Understanding APIs and picks up the words “where can…be obtained” as the user’s intent, and “Bin” and “Yellow” as entities (that could easily also have been “Green Bin” or “Rates Bill”, etc.). This invokes an interaction with the council’s Asset application and an order of the Asset required, and likely also any financials that go with it through the Billing system.

The question, therefore, leads to a booking and a delivery and bill; all without having to visit or call the council office and no on-hold telephone waits.

Who is our own Eva?

Eva
Eva – Exposé Virtual Assistant

It’s just been Christmas time, and Eva joined festivities 😊

If you browse to the Exposé website, http://exposedata.com.au/, you will meet Eva if you select “Chat with us now”. Eva was initially (Eva version 1) built to act as an intermediary between the website visitor and our knowledge base of questions and answers.  She is a tool that allows you to insert a series of questions and she returns answers. She learns from the questions and the answers using machine learning in order to improve the accuracy of responses. The net result is users spending less time searching for information on our website.

Eva version 2 was meant to solve our main pain point – what happens if the content on the web (or blog) site changes? With Eva version 1 we would have had to re-train Eva to align with new/ altered content. So, in version 2 we allowed Eva to dynamically search our WordPress blog site (this is where most of the content changes occur) so as to better answer user questions with up-to-date information.

And if the user’s question could not be answered, then we log this to an analytics platform to give us insight as to the questions visitors are asking.

Analytics
Eva – Analytics

In addition, we trained a language model in Microsoft Language Understanding Intelligent Service (LUIS) and built functionality inside of the Azure bot service to utilize functionality from the WordPress Exposé blog.

An example of an interaction with Eva can be seen below. As there are a few blogs that involve videos Eva will identify the videos and advise the visitor if there is a video on the requested subject.

EvaInteraction

Eva clearly found a video on predictive analytics on the blog site and so she returns a link to it. But she could not find anything on cats (we believe everyone loves cat videos 😊) and informs the visitor of this gap. She then presents the visitor with an option to contact us for more information.

Eva has learnt to understand the context of the topic in question. The answer is tailored depending on how the question is asked about “Predictive Analytics”. For example…

Chat

Go and try this for yourself, and try and replace “predictive analytics” with any of the topics below to get a relevant and contextual answer.

  • Advanced Analytics
  • Artificial Intelligence
  • Virtual Reality *
  • Augmented Reality *
  • Big Data *
  • Bot Framework
  • Business Intelligence
  • Cognitive Services *
  • Data Platform
  • Data Visualization *
  • Data Warehouse
  • Geospatial
  • IoT *
  • Machine Learning *
  • Predictive Analytics *

* Note that at the time of publishing of this article we only have videos for these topics. A comprehensive list of videos can be found here

Eva is ever evolving and she will soon become better at answering leading chained questions too.

GOTCHA: Eva was developed whilst the Azure Bot Service was in preview, but Bot names must now contain at least 4 characters.

Did this really help?

Often technology that looks appealing lacks a true business application.

But as you have seen from the example with Eva, we asked her about a video on a particular topic. Imagine using your intranet (e.g. SharePoint), data held in a Database or even an operating system as a source of information for Eva instead to interact with.

Authors: Chris Antonello (Data Analytics Consultant, Exposé) & Etienne Oosthuysen (Head of Technology and Solutions, Exposé)