2-way sysex synchronization

Discuss Lemur and share techniques.
Post Reply
oldgearguy
Regular
Posts: 315
Joined: 02 Nov 2013 11:19

2-way sysex synchronization

Post by oldgearguy »

I'm not really sure of a good title for this, but hopefully folks may benefit if they find it.

I had a goal to write an editor for a synthesizer. I needed to have 2 way communication - send edits to the synth and also pull back the current patch/info at any given time. There were 2 obvious roadblocks - one was the 256 byte limit on sysex communication and the other was a lack of detailed knowledge of how Lemur actually worked with respect to frames and script processing. I initially tried to get it all working using a lot of On Frame scripts, but I didn't have much success.

I could get around the 256 byte limit by sending short bursts of sysex to simulate pushing panel buttons, but there was always the problem of overflowing the synth's MIDI input buffer. I needed a wait/delay mechanism to allow the synth time to process incoming messages. I also needed a delay between sending dump requests for various chunks of data since I couldn't simply pull back the entire memory. The internal time variable was useful for counting and very short delays, but because a script only processed for a set amount of time, there was no guarantee that a script would process sequentially from beginning to end, especially when calling other scripts and trying to delay processing with a for/do-while loop.

Lemur 5 introduced MIDI clocks and I came up with a solution using a MIDI clock as kind of a master pulse to which everything else could be synced. The general approach can be summed up like this --

create a script that executes 'On Clock'. The script contains separate stages: a simple if (stage == 1) {do stuff then set stage = 2)} else if (stage == 2) {do other stuff, set stage = 3, etc}. The last stage would call clock_stop(7); to terminate the stage processing. Since the script triggers on a MIDI clock pulse, only 1 stage at a time will be executed and the delay time between stages can be controlled by the clock BPM and the on-clock beat divisor (32nd notes, 16th notes, 8th notes, etc).

Then, whenever I needed to talk to the synth, I would set the stage number (stage = 1;), start the MIDI clock (clock_start(7);), and save off the beat count (c7_beats = clock_getbeats(7);). Inside the On Clock script I check the current beat count against the saved beat count and as long as it is greater than 1 I fall into the if (stage == ) code.

In this way, I could now sequence my MIDI transmissions by putting sysex sends and retrieves in separate stages when I needed a delay, thereby giving the gear time to process and react to the messages. I gave the users a basic configuration option to adjust the processing delay time by creating a slider that effectively changed the BPM between 100 and 160 (the slider is scaled with some math so that a larger delay time on the slider calculates to a slower BPM).

OK, now I had a way to synchronize communications, but I ran into a second problem. I wanted the on-screen controls to reflect the current values of the current patch being edited. Simple enough - retrieve the sysex and use a script to update the controls. Kind of simple... I ended up creating a variable that I changed (ff = (ff+1) % 1024) at the end of my sysexin() routine. The On scripts triggered whenever ff changed and updated the controls appropriately. Originally I had ff be a simple flip-flop (ff = 1 - ff) with ff starting at 0, but at some point in time I think I was getting more than 1 sysex message in during a frame and therefore the ff value looked like it hadn't changed the next time Lemur evaluated it for script processing - again, I am not exactly sure how Lemur is doing all this, I am just coding to make it work given what I know. Also, at the end of the sysexin() I save off the current time value for later use.

If you've gotten this far without too much of a headache, you might see the problem looming. If I have a control (say a slider for oscillator frequency) and I send a sysex message when the user moves the slider so the synth is updated (On script triggered by any change to x) that works great. If I have an update script that changes x to match the incoming sysex data so the slider is positioned correctly, that may cause x to change which results in a sysex message being sent back to the synth. Not a huge deal you might say, except that the synth now thinks every patch has been edited when all you've really done is simply pulled it back to look at. Plus extra data is being transmitted when it's not necessary. To keep things cleaner, I had to implement a fix.

There's a 2 part solution to this problem. For things with a z value, I can check if z == 1 and if so, send the sysex since a user is actually touching the object. If z == 0, I know that x is being changed by incoming data and I don't send anything back. If I don't have a z, I need to do something else. Enter the saved time info from sysexin(). I send a sysex request down to the synth. Sysex data comes back. The time is saved. Controls get updated with the sysex. Non-z controls blindly call my sysex_send() routine to transmit the change. In the sysex_send() routine, I check the current time value. If the delta between the saved time and the current time is very small, I do not send the sysex. The rationale is that a user would have to be able to touch a control within 0.3 seconds (my current delta) of receiving a sysex to fool the system. Since a user probably isn't that fast, I can safely assume software is causing it and block the send.

I think that's enough for now. if you're writing similar editor code or just want to ask questions, feel free. I hope this technique may be of use to some folks.
Softcore
Regular
Posts: 1613
Joined: 04 Nov 2012 08:34

Re: 2-way sysex synchronization

Post by Softcore »

The trick with z has occured to me on many occasions of dealing with designing bi-directional feedback for custom midi outs or scripted CC outputs.

Much appreciated for sharing the "time" trick AND the midi clocks tricks - Im sure Im gonna need them sometime as ideas.
oldgearguy
Regular
Posts: 315
Joined: 02 Nov 2013 11:19

Re: 2-way sysex synchronization

Post by oldgearguy »

Softcore wrote:The trick with z has occured to me on many occasions of dealing with designing bi-directional feedback for custom midi outs or scripted CC outputs.

Much appreciated for sharing the "time" trick AND the midi clocks tricks - Im sure Im gonna need them sometime as ideas.
You are welcome. If you are brave, you can open up my TX-802 template and see the implementation details. The code is commented, but it would be challenging to parse out the good bits without an overview/roadmap. MIDI clock scripts start with the clock number: c6_ and c7_. On Frame scripts start with frame_. Sysex in/out start with sysex
Post Reply