Tuesday, June 6, 2017

Visible Feedback for Raspberry Assistant

Still playing with the Google Home Assistant that runs on one of my Rapberry Pi's. And while playing I realised I missed something. I am planning to build the Assistant into a dummy mannequin head or into a mirror. However when I speak the words 'OK Google' I do not have a visible feedback telling that the Raspberry Assistant is actually reacting, so I have no way to see if it was triggered.

What I need therefore is a led that goes on whenever the Assistant starts listening and goes off when the last sentences have been spoken by the Raspberry Assistant. That's a good way of getting a feedback.

The Google Assistant kit that was supplied by the Raspberry Magazine MAGPI does have a visible feedback in the form of a led. I am however using the Google SDK version which works different.

So switch over to the Magpi version you would say. However the Magpi version is triggered by a button and not by speaking to it and further it has a rather nasty voice when it does not understand something. Overall it is a lesser quality version as the Google SDK is.

So let's attach a led.

First step is to attach a LED to the Raspberry Pi.




The LED is attached to IO pin 18 and through a limiting resistor of 1K to ground. Thats all.
The Magpi version uses pin 25. So for avoiding future compatibility issues you could use pin 25. However make sure that you alter the software accordingly.

The Software Patch

The Raspberry Assistant is for the main part written in Python which is great for us, as this can be altered easily.

To adress the I/O pins of the Raspberry, what we need to do to control the LED, you will need to use the Python GPIO library.


There is however a small problem.

The Raspberry assistant runs from within an ENV. The GPIO library is not directly available in that ENV. So we first have to make it available.



Open the file-manager which can be found at the top-right part of your screen.



Goto  /home/pi/env/lib/python3.4/site-packages/


Make a new folder called RPi and copy the contents of  /usr/lib/python3/dist-packages/RPi to the new RPi folder.


The last step is to alter __MAIN___.PY so that it turns on the led when the conversation starts and turns it off again when it ends.
You can find this file here: /home/pi/env/lib/python3.4/site-packages/google/assistant


Double click on __main__.py and it will open in an editor. Now alter the file as in the source code below.  Do NOT copy the file below as it may have wrong indents.

WARNING:
Python is very picky about indenting.
The indents in this listing are SPACES not TABS !!!
If you mix the indenting the program will not work, so only use SPACES for indenting.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
#!/usr/bin/env python

# Copyright (C) 2017 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the 

"License");
# you may not use this file except in compliance with the 

License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, 

software
# distributed under the License is distributed on an "AS 

IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, 

either express or implied.
# See the License for the specific language governing permissions 

and
# limitations under the License.


from __future__ import print_function

import argparse
import os.path
import json
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
GPIO.setup(18, 

GPIO.OUT)
GPIO.output(18, 

GPIO.LOW)

import google.oauth2.credentials

from google.assistant.library import Assistant
from google.assistant.library.event import EventType
from google.assistant.library.file_helpers import existing_file


def process_event(event):
    """Pretty prints events.

    Prints all events that occur with two spaces between each new
    conversation and a single space between turns of a conversation.

    Args:
        event(event.Event): The current event to process.
    """
    if event.type == EventType.ON_CONVERSATION_TURN_STARTED:
        print()
        GPIO.output(18, GPIO.HIGH)
    print(event)

    if (event.type == EventType.ON_CONVERSATION_TURN_FINISHED and
            event.args and not event.args['with_follow_on_turn']):
        print()
        GPIO.output(18, GPIO.LOW)    


def main():
    parser = argparse.ArgumentParser(
        formatter_class=argparse.RawTextHelpFormatter)
    parser.add_argument('--credentials', type=existing_file,
                        metavar='OAUTH2_CREDENTIALS_FILE',
                        default=os.path.join(
                            os.path.expanduser('~/.config'),
                            'google-oauthlib-tool',
                            'credentials.json'
                        ),
                        help='Path to store and read OAuth2 

credentials')
    args = parser.parse_args()
    with open(args.credentials, 'r') as f:
        credentials = google.oauth2.credentials.Credentials(token=None,
                                                            **json.load(f))

    with Assistant(credentials) as assistant:
        for event in 

assistant.start():
            process_event(event)


if __name__ == 

'__main__':
    main()

Choose File/save and all is done.

Let's look at the listing.

23 import RPi.GPIO as GPIO
24 GPIO.setmode(GPIO.BCM)
25 GPIO.setup(18, GPIO.OUT)
26 GPIO.output(18, GPIO.LOW)

Line 23 to 26 are new entered lines.
First the GPIO library is imported. The GPIO mode is set to BCM so the IO pins will match the Raspberry's layout. And then GPIO 18 is defined as an OUTPUT and set in LOW state so the LED will initially be off.

44    if event.type == EventType.ON_CONVERSATION_TURN_STARTED:
45        print()
46        GPIO.output(18, GPIO.HIGH)
47   print(event)
48
49   if (event.type == EventType.ON_CONVERSATION_TURN_FINISHED and
50            event.args and not event.args['with_follow_on_turn']):
51        print()
52        GPIO.output(18, GPIO.LOW)

This is where it all happens.
As soon as the conversation starts GPIO 18 will be set HIGH (on) and when the conversation finishes the GPIO 18 is set LOW again (off).

Remember that if you have chosen to put the led on GPIO 25 you will have to later the source code accordingly.

That is all folks.

Restart your Pi and restart the Assistant


Now when you say "OK Google" the led goes on and when the conversation ends the led goes off again. A nice visual feedback that shows that you pronounced "OK Google" the right way so the Assistant understands.

OK, I am off for the summer hollidays.
New stories will be posted in August or so.

Till then: have fun !!!

Luc Volders