-
Notifications
You must be signed in to change notification settings - Fork 137
February 2021 Big Scenic Update RFC
February 2021
I've created an issue in the main Scenic repo for comments on this plan...
Also, this is all work in progress. Subject to change...
Scenic is a UI framework written in Elixir intended for use with small devices and apps. It's design point is control surfaces and relatively simple UI. Rendering can be done on simple machines support OpenGLEs 2.0 and higher and can be relayed for use on other machines.
Some goals of the Scenic include
- Must be understandable and relatively easy to program from Elixir
- Must render with fidelity across many computers and OS infrastructures
- Must be lightweight in terms of resources including power usage.
- Must be remotable / relayable to other computers
- Must be remotely usable over a 3G and/or BLE connection.
- Must tolerate relatively high latency between ViewPort, Drivers, and Rendering
- Must NOT require any open local Internet ports
There are more, but the point is that to date, Scenic has not fulfilled the goals around remote display and relay services. This is because we needed to better understand how the Kry10 Operating System works and passes messages between apps before designing this functionality. We will continue to support multiple platforms including Mac, Linux and Nerves. Hoping for, but not sure about, Windows for now.
The content of this document is currently being worked on in Scenic and is shared in the spirit of collecting feedback.
The current versions of Scenic use a 3-tiered model that has Scenes at the top, a Viewport layer that translates and normalizes graphs, and a driver layer that understands how to draw pictures. These drivers can also include serialization and sending data over the wire.
So far, these drivers independently decide how to interpret and encode the graph data. This typically means compiling it into a render script that is sent to a native code Port that does the actual rendering.
Starting with version 0.11.0 (It isn't 1.0 yet with good reason), Scenic will have several major changes to this model.
- Rendering a graph into scripts has turned out to be pretty much the same across all the drivers. Compiling binary render scripts will be standardized into a single format and performed in the Viewport layer. Drivers will then receive render scripts that they can pass directly to the render engines. This will dramatically simplify the drivers.
- The Path primitive is being deprecated and replaced with a Script primitive. The data for a script primitive is, literally, the output of the graph compiling engine.
- Multiple graphs can be transformed into scripts at compile time and can be compiled and sent to the
ViewPortby the same Scene. This effectively lets a scene break a complicated graph into smaller, more simple units. - Graphs are compiled into Scripts using an-immediate mode style API, which is also exposed to applications. This means you can build customized draw scripts to do what you want. This is what replaces
Primitive.Path.
One way to think of these changes is that the old model thought of the graph primitives as objects that the drivers understood internally. The new model presents the interface between the Viewport and the drivers as a virtual graphics API that can be driven from a script. Another way to say this is that the interface between Scenes and the Viewport is a retained-mode model. The interface between the Viewport and the Drivers is now immediate mode.
That description isn't quite right as the "immediate mode" commands are really scripts that are passed around. The script is the moral equivalent to the Graphics Command Buffer concept, except with higher level primitives.
Either way, there is now a greater distinction between the two interface layers of the Viewport.
The interface at the Scene level isn't changing very much. The interface at the drivers is.
When a graph is rendered, the output is a script, which is a list of calls to this virtual API. I am also exposing this API directly on the new Script Primitive, so you will be able to hand-roll your own scripts directly.
If this API reminds you of the Web Canvas API, that is on purpose. It isn't exactly the same (there are more high-level primitives), but everything here is can be rendered in a canvas.
This API is very much subject to change as I am coding it.
push_state() save() takes no parameters and saves the current transform and inherited style state. This state can then be restored using the restore() command.
pop_state() restore() restores the state previously saved by save().
scissor( width, height ) Set a scissor region that will clip subsequent drawing
line( x0, y0, x1, y1 ) The first point in the line to be drawn is defined by x0, y0 and the second point in the line is defined by x1, y1.
triangle( x0, y0, x1, y1, x2, y2 ) Each x/y pair represents a corner in the triangle.
quad(x0, y0, x1, y1, x2, y2, x3, y3) Each x/y pair represents a corner in the quad.
rectangle( width, height) Rectangles are defined by a width and height pair. Position them using transforms.
rounded_rectangle( width, height, radius ) Rounded rectangles are defined by a width, height, and radius. Position them using transforms.
sector( radius, angle ) Sectors are defined by a radius and an angle. The center point is always 0,0. Position it with transforms.
arc( radius, angle ) Arcs are defined by a radius and an angle. The center point is always 0,0. Position it with transforms.
circle( radius ) Circles are defined by a radius. The center point is always 0,0. Position it with transforms.
ellipse( radius0, radius1 ) Circles are defined by two radii. The center point is always 0,0. Position it with transforms.
text( utf8_string ) Text renders the given utf8 string. The origin is always 0,0. Position it with transforms.
script( child_script ) script renders a child script using the current transform and style state.
begin_path() Starts a new path to be rendered. Can be called multiple times in a single script.
close_path() Closes the current path. This is the same as drawing a line from the last drawn position to the first one. Joins are done correctly.
move_to( x, y ) Move the current draw position to x, y
line_to( x, y ) Draw an arc from the current draw position to x, y. The draw position is then updated to the new one.
arc_to( x1, y1, x2, y2, radius ) Draw an arc starting from the current draw position to x, y. The draw position is then updated to the end point.
bezier_to( cp1x, cp1y, cp2x, cp2y, x, y ) Draw an bezier curve starting from the current draw position to x, y. The draw position is then updated to the end point.
quadratic_to( cpx, cpy, x, y ) Draw an quadratic curve starting from the current draw position to x, y. The draw position is then updated to the end point.
solid( x, y ) Fill in the current path as a solid.
hole( x, y ) Cut out the current path as a hole.
scale( x, y ) Scale the subsequent drawing commands by x, y
rotate( radians ) Rotate the subsequent drawing commands by radians. The rotation is done around the 0,0 point. You need to combine it with the appropriate translation to get the same effect as a pin.
translate( x, y ) Translate the subsequent drawing commands by x, y
transform( a, b, c, d, e, f ) supply a 3x2 matrix to transform all subsequent drawing commands.
fill_color( color ) Sets the next fill to be the given solid color.
fill_linear( start_x, start_y, end_x, end_y, color_start, color_end ) Sets the next fill to be the given linear gradient.
fill_radial( center_x, center_y, inner_radius, outer_radius, color_start, color_end ) Sets the next fill to be the given radial gradient.
stroke_color( color ) Sets the next stroke to be the given solid color.
stroke_linear( start_x, start_y, end_x, end_y, color_start, color_end ) Sets the next stroke to be the given linear gradient.
stroke_radial( center_x, center_y, inner_radius, outer_radius, color_start, color_end ) Sets the next stroke to be the given radial gradient.
stroke_width( width ) Sets the width of subsequent strokes
NOTE: I haven't looked into the image fill API yet.
cap( :butt | :round | :square ) Set the line end cap style
join( :bevel | :round | :miter ) Set the line join style
miter_limit( :bevel | :round | :miter ) Control the max miter size
font( name ) Use the named font when drawing text
font_size( px ) Set the text size.
text_align( :left | :center | :right ) Set the horizontal text alignment.
text_base( :top | :middle | :alphabetic | :bottom ) Set the vertical text alignment.
Another important change, albeit one that won't visibly affect coding scenes, is that the layout of the supervision tree for scenes is changing significantly. The goal is to eliminate the driver restart storm that can happen when a scene crashes during initialization.
If you look at it in :observer, it won't look that different, except that scenes and drivers are managed as
children under a DynamicSupervisor that is in turn managed by a static :one_for_all that also
manages the Viewport. This means that if the Viewport crashes, everything restarts. But if a driver or
scene crashes, only the offending process restarts without bringing down the Viewport itself.
Input will largely work the same as before with one notable exception. Currently, when a positional event happens,
such as a click, mouse_move or similar, the Viewport walks the tree of all the graphs from all the
scenes - backwards - transforming the point according to the multiplied matrices at all levels in order to figure
out which scene/id to send the event to.
This can be a lot of compute work.
It is also largely unnecessary as the Primitives you usually want to see input events on are just a subset
of a total Graph. So, you will now need to use the new input: true style on primitives you expect input events
for. Anything not marked with :input will be skipped (or better yet, not included at all) in the walk-it-backwards
algorithm.
As part of supporting multiple Graphs-per-Scene and the rearranged Supervision tree, the machinery that manages the lifecycle of components is being re-written. The new version is much simpler.
But... I'm going to have to go back on a change I did earlier.
Originally, you sent a Graph to the Viewport using a push_graph() function. This was deprecated in favor
of returning something like {:noreply, state, push: graph} from the various handlers. This was good in the sense
that it prevented a Scene from spamming the Viewport with lots of pushes. But, it complicated things in other
ways, especially when scripts get involved.
So, I'm deprecating the {:noreply, state, push: graph} return value and going back to push_graph(). The spam
worry has been mitigated in the new driver model, so that is OK.
The Viewport is largely being re-written. I'm trying to keep the API familiar, but it wasn't a set of functions
that most Scenic apps called directly. This is in flux, but think that it adds facilities to handle multiple
graphs per scene, scripts, etc...
There are a smaller set of other changes happening. For example, several of the Primitives and Styles will be simplified as they contain extraneous data. They made sense at the time, but work on the render scripts as shown that there are further gains to be had. Some of these are Breaking Changes.
- Sector and Arc can be reduced to just a radius and angle. Rotation transforms are then used to reposition the result on an angle.
- Transforms are being reduced from 4x4 matrices (3D) to 3x2 matrices (2D). This is a conscious choice acknowledging that Scenic is a 2D graphics engine. This will (probably) result in a few new transform shortcuts, such as :flip and :reflect, but I haven't gotten to that yet.
- Join styles will be reduced to just Bevel, Round and Miter. This is to better match the joins available in the web Canvas API.