Google Cloud Speech to Text Voicemail Transcription

[junction1153]

Edit: Updated 04/02/2021

***As Mark stated in training videos, you should be using email queue for a production server. I've personally experienced issues with freeswitch locking up and completely crashing under heavy load. Root of the issue was transcriptions***

Make sure you have the latest fusionpbx github repo, at least for record_message.lua as of 10/2020

Step 1. Make sure you have a google cloud API generated with the proper permissions to use the speech to text engine

Step 2. Advanced --> Default Settings --> Voicemail. If a subcategory already exists please do not duplicate it. Change the values per bellow’s directions and enable the value.

dont forget to “reload” on the top right of the page when done


Add: Category: voicemail
Subcategory: transcribe_provider
Type: text
Value: google
Enabled: true

Add: Category: voicemail
Subcategory: transcribe_enabled
Type: boolean
Value: true
Enabled: true

Add: Category: voicemail
Subcategory: json_enabled
Type: boolean
Value: true
Enabled: true

Add: Category: voicemail
Subcategory: transcribe_language
Type: text
Value: en-US (or can be any one of these: https://cloud.google.com/speech-to-text/docs/languages)
Enabled: true

Add: Category: voicemail
Subcategory: transcribe_alternate_language
Type: text
Value: es-US (or can be any one of these: https://cloud.google.com/speech-to-text/docs/languages)
Enabled: true
***a value must be present for this. I have not yet had time to set the script where it skips it if there is no value. But basically with this value set, if google detects another language, it will transcribe in the alternative language provided instead of trying to transcribe in english)***

Add: Category: voicemail
Subcategory: google_url
Type: text
Value: https://speech.googleapis.com/v1p1beta1/speech
Enabled: true

Add: Category: voicemail
Subcategory: google_key
Type: text
Value: YOUR API KEY HERE
Enabled: true

This will only transcribe up to one minute of speech, per google's limitation. There is a way to transcribe longer, but well beyond my expertise and probably unnecessary for the majority of users.

Please let me know if you have any questions/comments/etc

 

[junction1153]

@markjcrane please feel free to use this for fusionpbx's code. I am a bit tied up the next few days but I will work on organizing the code and contributing to git early next month. Hopefully in time to use it in the email queue as well.

Edit: I just figured it all out and submitted the git contribution

Assuming you approve it, these are the directions that go along with it:

Advanced --> Default Settings --> Voicemail.
Add: Category: voicemail
Subcategory: transcribe_provider
Type: text
Value: google
Enabled: true

Add: Category: voicemail
Subcategory: transcribe_enabled
Type: boolean
Value: true
Enabled: true

Add: Category: voicemail
Subcategory: json_enabled
Type: boolean
Value: true
Enabled: true

Add: Category: voicemail
Subcategory: transcribe_enabled
Type: boolean
Value: true
Enabled: true

Add: Category: voicemail
Subcategory: json_enabled
Type: boolean
Value: true
Enabled: true

Add: Category: voicemail
Subcategory: google_key
Type: text
Value: INSERT YOUR API KEY HERE
Enabled: true

Add: Category: voicemail
Subcategory: google_url
Type: text
Value: https://speech.googleapis.com/v1/speech
Enabled: true

 

[powersofpain]

Wanna thank the work you guys have done so far on the Google Speech to Text transcription.

I just like to add my findings.
It will seems that 'alternativeLanguageCodes' option is not (no longer?) supported.
I keep getting 400 errors in the google console then i took a look at the record_message.lua file and did some console debugging.
It turns out that 'alternativeLanguageCodes': 'es' is the the culprit.

Removing this solve the problem.

 

[volkswagner]

Thanks for providing this info. I'm very surprised that this isn't a more popular topic. I was able to get
Google Speech API to transcribe voicemails on FreePBX using a tutorial. I can't find a similar
or any comprehensive how-to for using google's speech engine to transcribe voicemails with FusionPBX.

I have file > fusionpbx/app/scripts/resources/scripts/app/voicemail/resources/functions/record_message.lua
dated May 30 2021. I just don't know what other dependancies/steps I need to take.

Can @powersofpain or @junction1153 offer more details on how to get this working?

 

[junction1153]

At this point it should work right out of the box. Please follow step 2 in the original post to get it set up. It’s been updated with the current code. When you finally do get it working, i recommend you migrate to message queue if you have an active system. You WILL see Freeswitch lock up and crash otherwise. This is not specific to google, this is with any transcribe provider you end up with.

 

[volkswagner]

Thanks @junction1153

I added the seven entries/sub_categories to voicemail.
I see in the body of the email "Transcription".

Oddly during testing, I saw "Transcription ${message_text}
But now I'm not seeing that. I think this is because I blindly
added the seven entries, not expecting some already existed.

I deleted two duplicates "transcribe_enabled" and "json_enabled".

Hmnn,
I see those duplicates are in the second post above. I must need
both? I don't understand how these seem redundant/identical but
how/why the system uses them uniquely.

When I add the duplicate entries back, email doesn't send at all.

This is what I have for Default voicemail Settings:

display_domain_name	boolean	TRUE
enabled_default	boolean	TRUE
google_key	text	itsAbigLongSecret
google_url	text	https://speech.googleapis.com/v1/speech
greeting_max_length	numeric	90
greeting_silence_seconds	numeric	3
greeting_silence_threshold	numeric	200
json_enabled	boolean	TRUE
keep_local	boolean	TRUE
message_max_length	numeric	300
message_order	text	asc
message_silence_seconds	numeric	3
message_silence_threshold	numeric	200
not_found_message	boolean	FALSE
password_complexity	boolean	TRUE
password_length	numeric	8
password_min_length	numeric	4
remote_access	boolean	FALSE
smtp_from	text
smtp_from_name	text
storage_type	text	base64
transcribe_enabled	boolean	TRUE
transcribe_language	text	en-US
transcribe_provider	text	google
transcription_enabled_default	boolean	TRUE
voicemail_file	text	attach

Perhaps I need to copy some of these to my specifc domain?
Here's the settings for my domain.

Type	Value	Enabled
method	text	smtp
smtp_auth	text	TRUE
smtp_from	text	user@googleDomain.com
smtp_from_name	text	PBX
smtp_host	text	smtp.gmail.com
smtp_hostname	text	myHost.myDomain.com
smtp_password	text	***************
smtp_port	numeric	587
smtp_secure	text	tls
smtp_username	text	user@googleDomain.com
smtp_validate_certificate	boolean	TRUE

 

[junction1153]

Please make sure there are no duplicates. Also I’m updating the directions now. Mistake on the URL. Use this URL instead: https://speech.googleapis.com/v1p1beta1/speech

 

[volkswagner]

@junction1153 THANK YOU!

That worked a treat.
I updated the URL and all is well.

I really appreciate you taking the
time to document this and for
helping me directly.

Cheers!