A thought on improving voice user interface while ensuring privacy

Voice user interface is going to be one of the ways we interact with our devices as we go about our daily lives. It is just a very intuitive way for us because we communicate primarily via voice with text and images to complement.

But there still are various problems that need people to work on them to improve the overall experience. One of it is related to how the AI behind voice user interface can interact with us more naturally, like how we interact with fellow human beings.

A premium Medium article written by Cheryl Platz got me thinking about that. It also covered a little on privacy and why it is a contributing factor that make it difficult for current generation of AIs to speak more naturally and understand the context when we speak. Unless, companies don’t give a shit about our privacy and start collecting even more data.

In this article, I am going to share what I thought could help improve the AI and ensure user privacy.

Current Implementations and Limitations

What an AI needs to be better at understanding and responding in ways most useful to us are processing power, a good neural network that allows it to self-learn, and a database to store whatever it has learnt.

The cloud is the best way for an AI to gain access to a processing power and huge enough database. Companies like Amazon and Microsoft offer cloud computing and storage services via their AWS and Azure platform respectively at very low cost. Even Google offers such services via their Compute Engine.

The problem with the cloud is reduced level of confidence when privacy is involved. Anything you store up there is vulnerable, available for retrieval through security flaws or misconfigurations. Companies could choose to encrypt those data via end-to-end encryption to help with protect user’s privacy but the problem is the master keys are owned by said companies. They could decrypt those data whenever they want.

Or you could do it like what Apple did with Siri, storing data locally, and use Differential Privacy to help ensure anonymity but it reduces the AI capabilities because it doesn’t have access to sufficient amount of personal data. Two, Siri runs on devices like Apple Watch, iPhones and iPads, which could be a problem when it comes to processing and compute capabilities, and having enough information to understand the user.

Although those devices have more processing power than room-sized mainframes from decades ago, it’s still not enough, energy-efficiency and capability wise, to handle highly complex neural networks for better experience with voice user interfaces.

Apple did try to change that with its A11 Bionic SoC that has a neural engine. Companies like Qualcomm, Imagination Technologies, and even NVIDIA are also contributing to increase local processing power with energy efficiency for AI through their respective CPU and GPU products.

Possible Solution

The work on the hardware by companies should continue so that there will be even more powerful and energy efficient processors for AI to use.

In addition to that, what we need is a standard, wireless-based protocol (maybe bluetooth) for the AI on our devices, irrespective of companies, to talk to each other when they are near to each other and in our home network. This way, the AI on each of those devices can share information and perform distributed computing, thereby improving its accuracy, overall understanding of the user, and respond accordingly.

A common software kernel is also necessary to provide different implementation of neural network a standardized way of doing distributed computing efficiently and effectively.

So now, imagine Siri talking to Alexa, Google Assistant or even Cortana via this protocol and vice versa.

Taking privacy into account, information exchanged via this protocol should be encrypted by default with keys owned only by the user. Any data created or stored should only reside on device also encrypted and nowhere else. Taking a page out of Apple’s playbook, the generated keys should come from some kind of hardware-based “Secure Enclave”.

To further improve the neural network, Differential Privacy should be applied on any query or information sent by the AI to the cloud for processing.

Conclusion

The above is really just a thought of how current the AIs powering voice user interfaces can be improved.

At the end, it’s really up to the companies to decide if they want to come together and improve all our lives taking into account our privacy and security.

Advertisements

Thought Snapshot #2

Snippet #1

Being un-busy doesn’t mean not doing any shit. It simply means doing the things that matter, the things that are essentials. Everything else is just noise. Don’t clogged up your calendar.

Snippet #2

Alcohol is a useful substance that help me minimise or neutralise my neuroticism. It remove my automatic, excessive self-control and allow me to be freer, happier, more extroverted.

Snippet #3

People put money on-top of everything else. I put environment conservation on-top. Thus I recycle.

Thoughts Snapshot #1

Introduction

Every day, from time to time, I would have these thoughts or comments about the world that I will publish straight to my Facebook account. But since my Facebook is mostly private, I decided to consolidate and share these thoughts or comments with the world as a snapshot entry. Previously, I have always published them as collections of random thought snippets under the Journal category. Now, I felt that they should be in their own category on my blog. In some cases, you may find that they are more suitable as a journal entry but I prefer to keep them as pure as they come. And if I don’t write them down somewhere, they are lost forever until one day, if I’m lucky, I will remember them.

Snippet #1

Most software companies aren’t design driven. They always apply engineering thinking to everything. They are all about the tangible output, fast cash, and I think that is why these companies are not amongst the most valuable companies in the world. It is especially so of system integrators. They will remain that way until they shut down, go out of business or bought over. They don’t bring to the table solutions that cover the emotive aspect of how one might use a piece of software. It shows. Humans intuitively know whether tender loving care (TLC) was applied to a product they buy. Of course, there is nothing inherently wrong with being pragmatic, being all about the objectives but your end product could become way more valuable to the “normal” people you are selling your product to. It is the subjective experience that will bring the most money in.

Snippets #2

An ex-colleague saw me today and commented how much happier I look. I’m happier than ever before because I remain steadfast to my values, doing things and acting in accordance to my own value system. To the people around me, you will also get something tangible. E.g. if I’m happy at work, my output is consistent and stable. If I’m happy with life, you get more smiley face or more jovial actions from me. But if you step on any of my values, everything flips. So, to external parties, the equation is simple and very easy to balance.

Snippets #3

If you want to get a LV bag, do you complain about the price? If you want to get a Rolex, do you complain about the price? Apple products are the Rolexes or LVs of the technology world. You want it, then shut up and pay for it or don’t buy it.

Snippets #4

Want to do product design or design a great user experience? Make sure you have an enthographer or someone with background in anthropology giving advice.

Snippets #5

It’s perfectly alright if you want to spend up to 14 hours every day, Monday to Sunday working. But don’t complain after when everything else in your life starts falling apart and you feel extremely miserable. Like someone smarter, more hardworking, and tried stuff, compared to me, once said, “You pick up the boulder, you can put it down any time you want.”

Journal #164 – Minimalism, Gratitude, Patience, and 5% More

Singaporeans sure complain alot; I complain a lot. We all are miserable.

Let me be clear. I don’t know of the minute details that led them to complain a lot but I assume it has to do with them NOT doing the following:

  1. Simplifying their lives
  2. Be grateful for what they already have
  3. Have patience for things to happen
  4. Applying 5% more effort, more time, to achieve what they want.

Of course, you may get upset and ask: how I can assume that? You may just say: I am not you or him or her. I am stupid. I haven’t experienced enough.

The list goes on.

I do question myself: Am I right or wrong with my assumption?

All I can say is, I don’t know. But what do I know? I drew my current conclusion through my observations thus far. And I do know I am like that too. I didn’t simplify my life. I wasn’t grateful for what I already have. I wasn’t patient enough to wait for things. I didn’t apply 5% more effort, or more time on whatever I want to achieve.

It made me unhappy. It made me FUCKING MISERABLE!

Now?

I simplified my life when it comes to material goods and owning things. Having access is more important. I buy only when I truly need it, and after I have asked myself several questions and answered them.

It doesn’t mean I am not affected by advertisement and the emotions associated with getting new things. I personally love getting gadgets. In the past, I got new phones, MP3 players, Discmans, etc., just because I want them. And at this stage of my life, I love Apple’s products and wanted to get the latest and greatest (iPhone X, I am looking at you). I am afterall like you, a human too. However, I applied 5% more effort to change my mindset (I am a lazy person and enjoy things status quo, so it’s a struggle everyday) and accept the feelings I have, then I moved on.

Simplification also come into play on what I have to say next. There is one thing in Singapore that always grind parents’ gears. I am not a parent nor do I foresee myself getting married anytime soon. So I don’t know what is it like. But I see it everywhere and I do know what it brings. It brings unhappiness. It makes everybody in the circle fucking miserable. I see it in every parent’s face.

What is it? Education.

Parents in Singapore always compare with their friends and colleagues when it comes to their kids’ educations. They fret over the school their kids go to, how much tuition their kids should get, etc.

Why they do it?

They have good intention behind it. All parents want their kids to have a better life than they did. They applied their worldview of having more money and climbing the corporate ladder as having a better life.

But it has gotten to a point where it becomes fucking excessive, especially when it comes to tuition or expectations parents have of their kids. It put pressure on both ends, no doubt. Parents, wanting the best for their kids, send them to the best school and tuition centers, spending out thousands of dollars every month. It strained their finances, creating unhappiness. Kids have more homework, less play. They suffer from unnecessary anxiety. As kids, they tend to suffer more because they are unable to articulate out. They internalized it as just went “because my parent blah blah blah…, I have to do it.” When they fail, they blame themselves. That leads to the increasing number of children committing suicide. When that happens, parents will be putting the blame on themselves. If they have more than one child, the other children in the household will also suffer.

So my question is: Is it fucking worth it?

And that question applies to your finances, and your family’s happiness and wellbeing.

That lead me to gratitude. I have also applied gratefulness whenever I am aware of it. There are times when emotions run high, I forget about it or didn’t realize it. I am grateful that I am alive, have a family, and friends. I am grateful that my parents allowed me to explore what I like and dislike. I am grateful that I am not living in poverty and had a decent education. I am grateful that I had and have decent jobs (though I don’t like my current one, it just pays the bills), so that I can have some stability in life.

I have also tried to be more patient. I am an extremely impatient person. I like to see results the moment I applied some action. It is still a work in progress. But over time, I have learned to take it slow and enjoy the process. Just like right now. I applied patience to my writing. In the past, all I thought about was wanting to be maybe a New York bestseller or at least be the best science-fiction novelist from Singapore. And I couldn’t wait to get there. Now I know it’s not easy. And it will never be. It takes patience and hard work.

Both gratitude and patience can go hand-in-hand too. It applies to my family and friends. I am also grateful for the fact that Singapore has a decent, working public transport that can get me to most places and try to be patient when there is a train fault. No need for random outburst or public display of unhappiness. If you can’t wait any longer, accept the situation, move on, and find an alternative.

If you studied and applied stoicism, even at the bare minimum, you will be able to understand what I am talking about.

But most Singaporeans will just complain. I don’t think they have that patience anymore. I also don’t think they have the gratitude anymore. They are always looking for greener grass without wanting to do anything about it. Or they just blame the politicians. I know I do. But it doesn’t change anything. Now I know that. You can vote out the politician at the next election cycle but the real problem is YOU.

Has always been you.

I have come to acknowledge and realize that politicians deal with big picture. They have a grand plan of how to make Singapore a better place. Someone has to do that so that we as nation has a direction. It is up to us to implement. But have we been implementing? We may have but is it enough?

That lead me to the part on 5% more. 5% more effort. 5% more time.

The loudest Singapore complainers are who I assume to be lazy. You can get all defensive with that all you want or lie to me or make up some excuses. I don’t care. At the end, you could be lying to yourself to make yourself feel good and then continue to feel fucking miserable. So start asking yourself, what have I been doing and finding an answer to that.

From what I have observed, these people are NOT spending 5% more effort and time to be on the ground, focusing on making changes to their immediate environment or to themselves. Instead, they rather spend the energy and time doing something as meaningless as complaining.

It’s just noise really. No meaningful actions.

I’m pretty sure even entrepreneurs also say that complaining does nothing. Only by doing, you see result.

And what do I mean by doing? Here are some examples:

 

If you are a rail engineer, put in 5% more effort (even if you are not fucking paid for it), when it comes to train maintenance. How much is 5%? I don’t know the actual amount for you. For me, as a Software Engineer, 5% more simply could mean running an additional unit test on a piece of function before pushing out that piece of code.

 

If you are a parent, put in 5% more time to instill patience in your kids so that they grow up to be patient to wait for things to happen. You see, impatience is a major contributing factor to quality problem at the end. People just want result or get something done and over with at the snap of a finger. So they cut corners instead of spending 5% more effort, 5% more time or even both to deliver quality. Technology these days has made us even more impatient. We get instant dopamine rush or gratification from our smart phones. Instant notifications. Instant news. Instant feedbacks. So much so that we forgot what is it like to wait.

 

You see, if we on the ground didn’t even do things properly within our immediate environment or with ourselves, be fucking patient with the process, be grateful for what we have already achieved, how the fuck can we effect changes at other places.

For me, minimalism was hard. Gratitude was, and sometimes, is hard. Patience is hard. But I applied the same 5% more mindset to them. The end result is. I’m happier than I was. I’m less miserable than I was.

So I hope you can see that, it all starts with us, as an individual. I’m good now. Are you?

Journal #160 – New Apple Products

On 12 September 2017 (13 September @ 0100hrs, Singapore time), Apple hosted a keynote event at the Steve Jobs Theater in the Apple Park Campus.

I actually stayed up from 0100hrs to 0330hrs Singapore to watch the whole event live on my MacBook Pro.

At that event, they announced new products spanning four product categories: Apple Retail Store, Apple TV, iPhone and Apple Watch.

For Apple retail store, SVP for retail, Angela, announced that many stores will be upgraded to become lifestyle centers or town plazas in various part of the world for people to gather. New concept designs will also be introduced and implemented at the stores with around 100 planned by the end of the year. She also promoted the Today at Apple events, and the stores will be increasing number of seats, improving the audio, and 50,000 beacons will be installed at 400 stores across 30 countries. If you are wondering about what I mean by beacons, you can read more here.

WatchOS 4 was next to be announced and it features new watch faces, and existing functionalities are expanded. One such functionality is the heart rate monitor. It will be better at detecting issues such as atrial filtration, a potential deadly condition. Activity tracking is also improved to allow better switching of workouts, can download gym equipment data. Apple Music on the watch is also getting an upgrade, featuring automatic recommendations, multiple playlists, and a better UI.

Remember the rumor about LTE-enabled Watch? Turns out it’s true. Apple announced Watch series 3. The LTE-enabled watch is now able to handle phone calls, and stream music without the need for the iPhone. Other upgrades include ability to connect to the AirPods to listen to music. Instead of using an actual SIM card, the Watch uses e-SIM.

I am personally very excited about the LTE-version because it will allow me to go for workouts and listen to music without bringing my iPhone 7 plus out.

Apple TV now finally supports both HDR and 4K with its A10X processor. As part of the announcement, Apple also announced the expansion of the TV app for countries like Canada and Australia to include local television programs. Last but not least, 4K Contents from Hulu, Amazon, and Netflix will be included.

The biggest announcement was actually about the iPhones. Apple first announced iPhone 8 and iPhone 8 plus that use glass for phones’ bodies with color-matched, aerospace-grade aluminum band, featured True Tone display, and HDR. The cameras are also specially calibrated for augmented reality. The portrait mode is also upgraded with new lighting capabilities. The phones also come with better stereo speakers, and wireless charging. To power these phones, they use the new A11 Bionic processor with Apple designed 3-core GPU, Neural Engine for machine learning purpose and the M11 motion co-processor. These phones will be available for pre-order on 15 September at 1501pm GMT8 with iPhone 8 starting at SG$1148 and iPhone 8 Plus starting at SG$1308.

After that, Apple also announced the more “advanced”, more expensive iPhone X to mark the 10 year anniversary of the iPhone. As the rumors indicated, home button has been removed, uses Face ID instead of Touch ID, and rely more on 3D touch and gestures for most of the interaction with the phone. iPhone X also featured Super Retina display with HDR using OLED screen. The iPhone X also featured a wider display at 5.8 inches. SVP of Software Engineering, Craig Federighi, demoed how easy it is to use the iPhone X with iOS 11. Face ID is also demonstrated to be very fast. The phone will be available for pre-order from 1501hrs GMT8 with the phone starting at SG$1648.

Personally, I don’t see the need for me to buy the new iPhones because my current ones can serve me rather well for a foreseeable future. That’s the minimalist me talking and a part of me however desire to own the iPhone x.

Raising awareness about basic web security

Disclaimer: I am not a certified or experienced security expert but the issues raised in this entry has been repeated many times by many other security experts and are so easy to solve. Yet these problems constantly resurfaced.

Over the past ten years, our lives have increasingly move to the web. Unless you are a complete computer illiterate, lived in an extremely rural area and has no internet access, chances are you would have used computers and did some stuff online.

With that, security of any website that collect user information, including password, is of utmost important. At the minimum, sites that need user to login with some kind of username and password should use HTTPS throughout. Even HTTPS is not perfect. There should be additional security mechanism in place such as encrypting username and password even before you hit your web server.

I’m not going to claim that I am a security expert but there are just some things that irks me when it comes to the implementation of web security by some local sites.

So let’s take a look at some exhibits.

Exhibit A: VR-Zone

VR-Zone is one of the more popular tech site that comes with a forum. I stopped using the site ever since I noticed that they don’t implement HTTPS at all for their forums.

And to think that the login page is just a div modal on the forums. It’s obvious when you mouseover the login button, it shows at the bottom of the page.

So after you clicked it, the login dialog pops up.

Now let’s take a look at the source for the dialog box. Let’s see the section I’ve highlighted.

I’m confused. Is it because the password is md5-hashed onsubmit that the site owner thought HTTPS isn’t necessary? Even so, MD5 hash is obsolete in this day and age. I can just use any off-the-shell CPU and brute force the password once I have sniffed the traffic between the browser and the server. Without HTTPS make all that so much more easier.

Now let’s assume that the site owner wakes up and decides the whole site should use HTTPS.

But the protocol for the form action is hardcoded to use HTTP instead of using relative path. Using relative path would have automatically ensure the form submission use whatever the original protocol is.

Put it this way, humans make mistakes and are near sighted. Therefore, there is a chance that people forget to clean up their codes or that the site owner don’t give a shit. Besides, not all web developers are good at their job. And when they are rushing to meet deadline, security isn’t even one of the priorities. Then combine the fact Singapore is particularly famous for outsourcing instead of getting the right people for the job… See where I’m going with this?

I don’t even know how the site owner store user credentials or data. Since the “security” of the site is almost non-existent, I doubt the security of the backend components. Also, VBulletin is well-known for having a fairly long list of security issues. Some of my personal data had been leaked in the past as part of data breaches of forums that uses VBulletin. Here you can see the amount of security flaws or bugs found with the software.

I also found at least one feedback about this HTTP/HTTPS issue in the forum itself. So I’m not the only one.

Login Information under http not https

Since the post was made in April and there has been no response from the site owner. It looks like the security of user data is not the main concern. Also, I’m assuming the site is hosted in Singapore. If that’s the case, they could, and I mean could, be in breach of the Personal Data Protection Act.

Exhibit A: Hardware Zone

Hardware Zone is owned by Singapore Press Holding. I am expecting the security to be better than most sites hosted in Singapore.

The main page does not use HTTPS.

The forum does not use HTTPS.

When you hover your mouse over the login icon.

This is what you get. You will be redirected to a HTTPS page to do login.

Since the main site is not protected by HTTPS, it would be relatively easy to modify the page content before it is send to your browser. Some malicious person may just decide to change the login URL and present you a lookalike login form. You enter your login credentials and that’s it. Considering that Hardware Zone does collect personal information such as your age, address, name and many more, if hackers do manage to get your login credential stolen, well, it means problem for you.

The login form itself is protected by HTTPS.

Let’s look at the source code.

At least they used relative path for the form action. Looks like the MD5 hashing is a default thing for VBulletin. But still, MD5? It’s time to seriously look into SHA-256 or SHA-512.

After you login however, you are redirected back to HTTP again. Even when you access your user profile, it’s on HTTP too. This is like a half-hearted attempt to do HTTPS. Shame on you SPH.

Thoughts about Microsoft Build 2017

Well, I’m late to the party. Other news sites, bloggers, and vloggers have all covered it. They have their own thoughts about it. I’m going to do my own take on it, on only features that I’m interested in

For those who don’t already know, Microsoft held an event in Seattle between May 10 to 12, 2017 that introduced to the world a whole bunch of new things for Windows 10 that we all should be excited about — by that I mean techies.

To summarize, during the event, Microsoft pointed out several trends that’s going on that most of us already know: the rise of IOT devices, Artificial Intelligence, Serverless Architecture. They also talked about what they are doing regarding those trends. They are digging in deeper, delivering better solutions or services that customers need.

I’m more interested in their new Fluent Design. User experience and graphics is important to me (though I can’t really draw very well). Microsoft has attempted over the past decade to deliver a nice interface. They introduced Aero with Windows Vista, which was a massive flop. It was slow and sluggish. I’ve personally used it and don’t really like it. Then came Windows 7 and they improved Aero. It was much snappier and still look great.

Then came Windows 8 where they introduced what was known then as Metro design, subsequently known as Modern UI and then Microsoft Design Language. I personally liked it more than the pseudo-3D of Aero in Windows 7 because it is simplistic and minimalistic. It was improved Windows 8.1 and then in Windows 10.

However, the biggest gripe about Microsoft’s Windows was that the user interface isn’t consistent. There are some glaring defects that any good user interface designer should pick up on and fix it. Then, there is the fact that there are thousands if not millions of applications that runs on it. Many of which are legacy application and their user interface hasn’t been updated in ages. On the other hand, the web user interfaces have gotten so much better though every webpage is starting to look like each other due to the extensive use of common frameworks like bootstrap.

With Fluent Design, Microsoft aim to make Windows look great. As a design language, there are five main features:
1. Light
2. Depth
3. Motion
4. Material
5. Scale

You can read more about the Fluent Design System here.

When you combine all five of them properly in your application, the experience the user get will be so much better, richer and immerse them. This design language is in part, I believe due to the increase use and popularity of mixed-reality devices.

As someone who has been using Apple for more than year and already buy into the ecosystem, I’m somewhat excited to see how future applications made for Microsoft Windows will look like and how it will affect the way we as users interact with our devices. My concern was that, there is always the issue where the third-party developers do not ensure their app is consistent with the overall look and feel of the Windows running on the user’s computer.

Right now, the Fluent Design System is implemented in a somewhat beta form with the insiders build of Windows 10. If you are interested, you can download that and try it. Until it is released fully and used by many of the applications in Windows, I will stick with the experience I’m getting from Apple’s ecosystem.