Patcher on another platform?

lodevalm 2017-09-19 18:50:23 UTC #1

Hello,
I'm a software developer and musician and a question came to my mind, just for curiosity: does the patcher communicate with the core as a service consumer or is it linked and use it as an external library?
(I didn't take a look at the code yet)
So, could be possible porting the patcher to a js platform an distribute it with electron? Does it make sense?

toneburst 2017-09-19 19:12:50 UTC #2

I'd love to see a JavaScript/CSS UI for the Axoloti Patcher, as I think it would solve a lot of the layout issues with the current implementation, but I have a feeling the code wouldn't be easy portable from Java.

a|x

orbitbot 2017-09-19 19:17:37 UTC #3

Although I don't have an exact answer, I've been thinking about something similar to be able to improve the patcher UI in a significantly faster manner. Documentation regarding the source code isn't really up to date AFAICT, so it's difficult to determine how anything works without digging in and reading through stuff, which therefore has taken lower priority b/c of other RL concerns.

From a feasibility standpoint, if at all possible, I wouldn't (at least initially) spend time trying to port any of the fundamental code to another language, rather just run uploading/patching commands from child processes. If a CLI-compatible interface doesn't already exist, it can probably be written faster than reimplementing the whole thing.

thetechnobear 2017-09-19 20:32:22 UTC #4

the current development is moving toward MVC, which will help separate the UI code from the core functionality, this could make it easier for someone to develop a 'service based' interface, though you'd still need to have the 'server' running java.

personally, Ive more interest in exposing some functionality which is useful for interacting with the boards... e.g. uploading patches, parameter interaction etc, which could be done the binary protocol that is used for the usb connection, and so is language independent.

lodevalm 2017-09-19 20:38:24 UTC #5

I've cloned the axoloti repo and did a super fast sight on it.
I understand the java part while I did only read some part of the firmware (actually I've never worked on firmwares and I know nothing about them, the only thing I can say is that Johannes Taelman is a sort of half-god fallen on heart and his code looks beautifull and self explained).

I could be wrong but all the c code for firmware and os specific shell task are on the axoloti runtime while the java code for the patcher is on the app.

So the app is responsble not only of the UI but also for the communication to the device and compilation of the patches and firmware.

Finally I think it is not possible to separate UI from serial operations and firmware compilations without a huge amount of refactoring...

thetechnobear 2017-09-19 20:48:23 UTC #6

not quite sure what you mean by that...
the Axoloti runtime only contains the GCC cross compiler, and a few other 'tools', to create a build environment, nothing else... its not related to the patcher really, and you could install these separately ... is really just a packing of stuff, that 'normal' users don't have installed on their computers. (e.g. MinGW for windows)

the patcher java code has (kind of) five separate functions:
- the patcher you see, which builds a 'model' (called patch)
- the code generator, which goes thru the model, taking each object, extracting the c code, and then following the wires to make a call chain , writes out xpatch.cpp
- run the build .. this is pretty simple, just calls make (various flavours for each platform)
- upload patch... sends patch via usb over a wire protocol
- live mode, which gets/sends messages via wire protocol about the parameter values changing

obviously this is a gross simplification, but that's basically what the patch does.
(and yes, the devil is in the detail)

hopefully with the MVC model these will be better abstracted, so it will be a cleaner division between these elements.

as it uses xml for patches/objects, and a binary protocol between the board, these are the most obvious entry points into doing something 'standalone' from the patcher code.

strum 2017-09-20 03:39:20 UTC #7

This was my thinking also.

How does the USB connection work and is there a sort of API for it?

What files in the firmware code deal with the USB communications?

orbitbot 2017-09-20 05:57:14 UTC #8

Yeah, this is exactly the kind of interface that would be needed to be exposed for f.e. standalone CLI usage and therefore also other programs. FWICT an alternative editor would have the responsibility of generating the patch XML files and matching those to some kind of UI.

thetechnobear 2017-09-20 11:10:24 UTC #9

USB protocol - check USBBulkConnection.java on patcher side, on the firmware side, its in a few places, but look for writing to BDU1. .. no there is no API.

not really, this would only cover one (or two) aspects of the problem space. it still doesn't over code generation/ compilation.
anyway, i think creating an alternative editor that is integrated is far from trivial.

I don't think that Java is really the 'bottleneck' (for JS etc to 'solve') for user requests - often its the underlying representation, or how we want things to be represented to the user - or to produce efficient code for the MCU... all pretty independent of implementation language.

as for 'faster development' , it was decided to have a longer 'development phase' to deal with some of the code issues that build up (technical debt in agile terms) , so the next version is much better structured, which hopefully allows changes to be made quicker (without fear to other bits breaking)

personally, Id urge developers to collaborate in the current development.
build an understanding of what is there, and why... then you can influence its direction.
e.g. part of the reason for moving to MVC is to allow an alternative view... so perhaps once that effort is complete, you'll be in a position to realise a JS type interface - if you still think thats a good idea - but thats only practical once you have a deeper understanding of the code base ... and like all programming, thats achieved by 'doing'

(as an open source developer, this is generally the approach I take, and also advise... even if I spin off projects later)

however, one aspect of open source is aiding alternative implementations, visions and ideas. so I look forward to seeing what you guys can produce.

strum 2017-09-20 11:30:15 UTC #10

Thanks, that gives me a place to start.

not trivial at all I had thought, but a control interface that could load patches and have a GUI for exposed parameters doesn't seem to difficult.

This is what I had thought also.

Very sensible, better to take time and design a system that works efficiently and allows for more flexibility for further development and expansion than just tinker with small details in a rapid cycle within the existing structure.

what is MVC?

I'm getting started on this but it's going to take at least a few months given my time availability and the complexity of the project. This gives me a good place to start anyway cheers.

orbitbot 2017-09-20 11:56:10 UTC #11

MVC stands for Model-View-Controller (wikipedia has a decent overview), and depending on the context may imply something very specific or something a bit more abstract, as there are some permutations on the acronym that imply other implementations. In any case, it's probably an indicator of improved architecture, modularisation and single responsibility for the classes involved.

Java itself is not really a problem as I see it, my personal problem is that I have low interest and realistically less time to invest in learning Swing or whatever else the UI framework is to solve some usability issues. F.e. on mac new floating windows always open really small and need to be enlarged, when searching in the object selector pressing down won't switch focus to the filtered list to go through the options, the object selector window cannot be resized so you need to scroll a lot to read the full descriptions etc.

Agree.

I don't have an understanding of the correlation between the UI elements, the axp files (that seem to be XML and not the binaries to load to the board), but the alternative UI would essentially do the above and then start child processes to compile and install etc.

So, if a CLI exists or can be made available to control the runtime parameters on the board, compile and install patches and whatever else exists, the building blocks for alternative UIs are already available.

toneburst 2017-09-20 12:53:54 UTC #12

I really think, from a useability standpoint, it's the GUI that's holding things back. That's understandable, but a shame.

Most users will have no interest in the underpinnings of the whole application, but if the code is structured in such a way that changes to the UI cannot be made easily and quickly, based on user feedback, then it doesn't reflect well on the whole project, unfortunately.

I'm sorry if this sounds harsh, but I think it's fair.

a|x

strum 2017-09-20 12:58:04 UTC #13

I'm afraid I have to agree.

Regardless of the rightness or otherwise of the attitude people do judge software on how it looks.

I would argue that the appearance and layout of the GUI has a lot to do with usability of the software as well.

I think people are probably put off Axoloti by how the gui looks.

toneburst 2017-09-20 13:39:00 UTC #14

I realise I've effectively trolled this thread, and I apologise for that.

My intention was not to hurt the feeling of the fine people who make and contribute to this software, and I'm not saying that the UI as it stands is in any way unusable. It's just it looks.. basic.. and there are a few long-standing glitches in terms of the way common functions work that niggle while using the Patcher.

Any changes to the code that would help separate the back-end code from the display parts, and facilitate tweaks to the Patcher UI, without risking screwing up other parts of the code has got to be a good thing.

Also, if anyone with design and UI (but not necessarily low-level coding) experience wants to get involved in redesigning or re-skinning the Patcher UI in the future, the above would help enormously.

It looks like this is being worked on, anyway, and I can only applaud that effort. I think in the long run it will pay off in terms of usability, and user-satisfaction.

a|x

lodevalm 2017-09-20 14:29:47 UTC #15

My first post in this thread was about changing UI code and eventually change language/GUI framework.
But I also think that Java is good (easily portable over operating systems, for instance I've installed the app on a linux on my job machine and on a mac that I use to play).

I agree with everything you wrote and to be a little OT, this conversation with you guys is more than welcome in a world full of haters, trollers and so on (even in the opensource community...)

Back to the topic, I can say that the interface is really not too bad (for instace puredata is more ugly...).

My primay job right now is making back-ends and front-ends working together but I did a lot of interaction design in the past so I know how UI works.
Please keep posting, because I'm very intrested in being involved in the project!

thetechnobear 2017-09-20 16:05:33 UTC #16

Yes, the move towards MVC (or rather something like it) , is about separating UI code from other 'functions', this abstraction will help in lots of different ways.

I was not trying, at all, to defend what the UI looks like, nor debating if the looks put people off (or not) ... it definitely could be improved, and you may be correct about being off-putting....

Im only trying to give an indication into the complications of writing a new UI... since I'm not sure how many here, have really looked at the code in depth.

you mean like a control surface... this will hopefully become easier soon, for axoloti control there is a necessity that the parameters can be self describing and so used without the patch file. this will also be useful for other controllers. ... this is a direction I will be exploring

(reading the axp file is actually quite involved, and not something i would undertake lightly... its not so much parsing it, but then 'understanding' it, and getting all related information.. and some information is just not there e.g. how types get parsed from an int32, into something like frequency)

This I totally understand... Ive a preference for C++, but I still contribute to java based projects to solve issues/improve things, because its a lot quicker than rewriting everything, even to get to feature parity would be (many) months of work, which I don't have time for. so I go with the flow, and try to incrementally improve things
( a bit different professionally where I'm responsible for the design)

but hey, I'm not discouraging anyone, dive in... Im sure you have a clear vision of what exactly you want to achieve, and how you plan to do it... often the case is, we only know by trying, and learn along the way.

as for me. too many projects, too little time ... so I'll just continue to help Johannes.

toneburst 2017-09-20 16:10:41 UTC #17

It's absolutely hideous, so yes the Axoloti Patcher is excellent in comparison!

a|x

johannes 2017-09-28 20:54:27 UTC #18

I'm not against such a move, but I believe it is most important to do such step by step.

The first step is separating program logic from the GUI code, so other GUI's can be "plugged in". The experimental branch has this largely in place.
Next, transitioning from a Swing GUI to a HTML5 GUI, I believe there are two options, using JFXPanel or chromium via jxbrowser, while still keeping the program logic in java.
Eventually the program logic could be converted to JS too, using Electron, but I'm not sure there is a substantial benefit here.

Perhaps one "toy" area where a html5/js-based gui could be integrated and explored is the "memory viewer" (in the experimental branch). A user-editable script could visualize memory data in arbitrary ways (like graphing).

My first concern however is turning the experimental branch into a stable 2.0 release.

lodevalm 2017-10-03 11:03:05 UTC #19

@johannes I completely agree.
Reading the code, I answered to myself that porting to a java API service with an electron client is too late (and maybe useless) considering the state of the project.
I'm not against java but swing hurts a bit myself

I've tried to fix a bug on the UI on the experimental branch: If you click the label to change name of an object then you drag the object itself, the editing field doesn't disappear.
I didn't solve it yet, while I found the place where it loose the focus and didn't get notified on the AxoObjectInstanceViewAbstract.

I'm a bit lost in the project structure...so solve this kind of things is useful for me to understand how you organized and named the classes...

Let's see if I will get some results...

johannes 2017-10-05 21:19:01 UTC #20

Two things seemed to cause this issue: dragging the object title bar didn't pull the focus away from the object instance name edit field, and removing the edit field component did not trigger a repaint.
I (think) I have addressed both in this commit.
Project structure... yes there is room for improvement. Two conventions are mixed:
xxxDefinition/xxx
xxx/xxxInstance
the first corresponds with something that belongs to the definition of an object, the 2nd to something of an instance of an object in a patch.
Further, everything referring to piccolo is currently broken.
A reorganization of packages and class names would make sense but I think I prefer to wait with this as it will make it hard to track code changes over such a reorganization.