Bug Fix and Documentation Updates

Fixed

• Loading pages from sub-folders without adjusting the uibuilder.start() parameters was failing. Now fixed. See also FAQ: uibuilder - multiple pages from 1 node.
• Accidentally made use of a node.js v14 feature (Optional Chaining) but base version is specified as v12. Changed code to v12 compatible until v14 becomes the base.

Changes

• Tech Docs sidebar menu:

  • Link to beginners walkthrough document added to Tech Docs sidebar.
  • Add link icons to external pages.
  • Remove redundant link to GitHub project page.

• uibindex page:

  • Changed to use new CSS (light/dark enabled).
  • Configuration Files section needs updating.
  • Further improvements and tweaks.

• Tech Docs updates

  • NEW PAGE: Configuring uibuilder nodes [incomplete]
  • NEW PAGE: Creating Templates [incomplete]
  • Updates for Configuring uibuilder page

Bug fix release

Fixed

• When no settings.js:uibuilder property exists, initial report doesn't have ip/port. Also get (node:2888) [DEP0118] DeprecationWarning: The provided hostname "undefined" is not a valid hostname, and is supported in the dns module solely for compatibility. as well as http://null:undefined or http://localhost:undefined/

  Fixes applied to uibuilder.js and web.js

v5.0.0 - The big one!

WARNING: v5 is a very major release and WILL break some deployments. Please check through the changes before deploying. In most cases, opening each uibuilder node in the Node-RED Editor, clicking Done and re-deploying will be enough.

BREAKING

• Clear client cookies - You may get some odd results if you don't because cookie handling as been significantly reworked.

• Installation of packages for use in your front-end code has been moved from the userDir to uibRoot.

  Note that only packages installed into the uibRoot folder will be recognised.

  Unfortunately, this means that you will need to re-install your packages in the correct location. You should uninstall them from the userDir.

  By default: userDir = ~/.node-red/, uibRoot = ~/.node-red/uibuilder/. However, both can, of course, be moved elsewhere.

  uibuilder will automatically create a suitable package.json file in uibRoot. That file not only lists the installed packages but also has a custom property uibuilder that contains metadata for the uibuilder modules. Specifically, it lists all of the necessary detailed data for the installed packages.

  The files <uibRoot>/.config/packageList.json and <uibRoot>/.config/masterPackageList.json are no longer used and may be deleted.

  You can now install not only packages from npmjs.com but also from GitHub and even local development packages. @scopes are fully supported and versions, tags, and branches are supported for both npmjs and GitHub installs.

• Removal of Security Processing - The built-in security features of uibuilder have now been removed.

  This is because I was not getting any closer to a design that was robust and safe and it was having a serious impact on development.

  The recommendation is that you use external security processing via a reverse proxy service (e.g. NGINX). I will be trying to provide some documentation for this.

  Alternatively, you can still provide security processing by making use of the various middleware features - see the <uibRoot>/.config folder for templates.

  Because of this, the security settings part of the Editor config panel has been removed. This included the use security flag, session length, JWT secret and auto-extend flag. If you have been using the vNext development branch, the Security tab has been removed.

• Peer installation of VueJS and bootstrap-vue yet again removed. Since these now need to be in the uibRoot folder which
  we don't necessarily know at preinstall time.

  We will be looking at another alternative method. Now that template switching is even more powerful and so is
  package management, it is likely that we will build something into the template installation process.

  Until then, please install the vue and bootstrap-vue packages via the uibuilder library manager if you need them.

• .config folder templates - In previous versions, any files in the master templates for the <uibRoot>/.config folder would be copied when Node-RED (re)starts - however, they would not overwrite existing files. From v5, the master .config templates have been renamed to end with .js-template and these will now ALWAYS overwrite whatever is in the <uibRoot>/.config folder whenever Node-RED (re)starts. That way, you always have easy access to the latest templates.

  In addition, some of these templates have significant changes and you should review the new templates before going live with uibuilder v5. You can (and should) also safely delete any .config/* files you are not using.

  In particular, the masterPackageList.json and packageList.json files are no longer needed or used and should be deleted.

• URL cannot be "uibuilder" - As this url is now used by various services, allowing it would potentially create name clashes and hard to debug errors. If you have an endpoint named "uibuilder", please rename it to prevent problems.

• Clear any old cookies - Cookie handling has changed for v5 (see the details below). New cookies are clearer and last only for the current browser tab session. You should clear out any old cookies that have been saved by your browser for each uibuilder app URL.

• Minimum Node.js version supported is now v12.20. Minimum browser version remains the same and must be one that supports ES6.

• New Dependency - I've finally given in and included the socket.io-client package in the dependencies. The reason for this is that the server does not correctly include the client package as expected. While it works if you are including the client using a <script src="..."> line in your index.html, it does not work if you are using web components or a build step with an import statement. So in that instance, you should use.

  Note the change of path when writing your HTML:

  <script src="../uibuilder/vendor/socket.io-client/socket.io.min.js"></script>

  Though the old path works as well.

  However, if you want to use uibuilder with web modules or with a build step and wish to import socket.io client, you must not load via a script tag. Instead, in your JavaScript or framework code:

  import { io } from '../uibuilder/vendor/socket.io-client/socket.io.esm.min.js'

  Better still, use the module compatible version of the uibuilderfe client uibuiderfe.mod.min.js or uibuiderfe.mod.js which will import the correct socket.io client for you.

New

• New layout for the Editor panel

  This is a much cleaner and clearer layout. It also blocks access to parts of the config that don't work until a newly added node has been Deployed for the first time so that its server folder has been created.

  There are also some additional error and warning messages to make things clearer.

• New node uib-sender - this node allows you to send a msg to any uibuilder instance's connected front-end clients.

  That means that it is pretty much the same as sending a message directly into a uibuilder node.

  You select the instance of uibuilder you want to use by selecting an existing uibuilder URL from the dropdown.

  You can also select whether you want input messages to go straight to the output port as well.

  Or, more usefully, you can allow "return messages". This allows a front-end client to send a message to node-red with some pre-defined metadata added that will route the message back to the uib-sender node. In this way, the sender node can be used as a semi-independent component.

  Note that this same method can be used by ANY custom node, check out the code to see how it works. It requires the use of an external, shared event module @TotallyInformation/ti-common-event-handler. The msg metadata looks like: { _uib: {originator: <sender_node_id>}, payload: ... }. The sender node id is just that, the Node-RED node id for the sender node instance.

  The uibuilderfe.js library has been updated to allow easy use of the originator property for uibuilder.send(). See below for details.

  There is a new page in the Tech Docs on using the sender node.

• New node uib-cache - this node allows you to cache input messages in various ways and recognises uibuilder's
  cache control messages so that a client browser (re)connecting to a web page will automatically get a copy of the cached pages.
  See the Tech Docs for details. An example flow is included in the uibuilder examples library.

  Note that you can use this node without uibuilder if you want to.

  There is a new example flow demonstrating the use of the cache node.

• New Feature Msg send middleware. You can now add a custom middleware file <uibRoot>/.config/sioMsgOut.js. The exported function in it will be called every time any msg is sent from any uibuilder node to any connected client. Please see the template file for more details. This rounds out the ExpressJS and other socket.io middleware (connect and on msg receipt) and helps make up for the removal of the uibuilder security features by allowing you to create your own bespoke identity and authorisation processing.

• New Feature Instance API's. You can now define your own API's to support your front-end UI. These run as part of the Node-RED server and can be called
  from your UI, or indeed from anywhere with access to the Node-RED server's user endpoints.

  You can add any number of *.js files to a folder <uibInstanceRoot>/api/. Each file will be loaded into the uibuilder instance and tested to make sure that it contains either a single function or an object containing functions who's property names match either an HTTP method name (get, put, etc) or the generic use.

  Such functions are added to the instances router. See the Tech Docs for more information on how to use the instance API's.

  Note that, because such API definitions reside in a potentially user-facing folder and may be significant security risks, their use is controlled by a flag in Node-RED's settings.js file uibuilder.instanceApiAllowed. This must be set to true for API's to be loaded.

• New Feature - Added a version checker that allows uibuilder to notify users if a node instance must be updated due to a change of version.

• New Feature - A default CSS style sheet has been introduced. Either include in your index.css file as @import url("./uib-styles.css");. Or in your index.html before the reference to ./index.cssas`.

  Currently this contains some :root classes defining colours and a switcher that picks up whether your browser is set to light or dark themes.

  It also has a number of classes that style the toast notifications if you are not using bootstrap-vue.

  Add th...

Minor Release

New

• Issue #151If the advanced option to "Show web view of source files" is selected, also show a link to the webpage.

Changed

• Issue #149 If security is turned on, you can now run without Node-RED using TLS even in production. This is because you may wish to provide TLS via a reverse proxy.

  You still get a warning in the editor though.

• Moved back-end libraries from nodes folder to nodes/libs to keep things tidier (especially if additional nodes added in the future)

• Add simple debug function to web.js to allow the ExpressJS routing stack to be dumped to stdout

Fixed

• Issue #150 Switching between src and dist folders now works without having to restart Node-RED. Existing routes are removed first then re-added.
• Common folder is only served once (previously it was been added to the ExpresJS router stack once for each node instance).

Small feature release with bugfix

New

• Add drop-down to adv settings that lets the served folder be changed between src and dist. #147

  • If the <uibRoot>/<servedFolder> folder does not exist, it will be silently created.
  • If the <uibRoot>/<servedFolder>/index.html file does not exist, a warning will be issued to the Node-RED log & the Node-RED debug panel.

• Allow front-end code to update the msg. #146

  This allows your front-end code to be its own test harness by pretending that a msg has been sent from Node-RED. It would also let you have a single processing method even if you wanted to use a non-Node-RED data input (e.g. a direct MQTT connection or some other API).

  uibuilder.set( 'msg', { topic:'my/topic', payload: {a:1, b:'hello'} } )

  When using this feature, the uibuilder.onChange('msg', function(msg) { ... }) function is still triggered as expected.

Fixed

• #148 Editor node config cannot escape https check when not running in development mode

Minor bugfix

Fixed

• Minor bug stopping the logoff msg processing from working.

Updated

• All dependencies and dev-dependencies updated

Major release

Major Changes

• Node.js v12+ is the minimum supported environment for Node-RED.

• Only "modern" browsers are now supported for both the Editor and the uibuilderfe front-end library as ES6 (ECMA2015) code is used.

  Let me know if this is a problem and I can build a backwards compatible version.

Template handling is significantly changed in this major release

New instances of uibuilder nodes will only be given the "blank" template which uses no front-end frameworks.

You can load a different template using the "Template Settings" in the Editor.

Loading a new template WILL overwrite any files with the same name. A warning is given though so even if you press the button, you can still back out.

You can choose from the following internal templates:

• VueJS & bootstrap-vue - The previous default template.
• Simple VueJS - A minimal VueJS example.
• Blank - The new default.
• External - See below.

But, you can now also chose an EXTERNAL template! This will let you choose from any remote location supported by degit. You can use TotallyInformation/uib-template-test as an example (on GitHub).

NOTE: When using an external template, no check is currently done on dependencies, you must install these yourself. I will try to add this feature in the future.

Changing the uibRoot folder

You can now set uibuilder's root folder - that stores configuration, common, security and each node's front-end code - to a different location. The default location is in your userDir folder in a sub-folder called uibuilder. If you are using projects, the sub-folder will be in your projects root folder. See docs/changing-uibroot.md for more detail.

Updated

• Update fs-extra to v10. No longer supports node.js v10, requires v12+.

• Make some class methods private in web.js and socket.js. Requires node.js v12 as a minimum as it uses an ECMA2018 feature.

• web.setup and socket.setup can only be called once.

• Socket.IO updated from v2 to v4.

• Added Admin API check for whether a url has a matching instance root folder. (Was an outstanding to-do)

• Reworked the info block that is printed to the log on startup. Much neater and with added info on the webserver being used.

• Technical Docs have been improved in line with some other work I did recently on enterprise standards.

  The docsify configuration has been greatly improved with a new theme and some automation for dates and document front-matter.

  Added a new page on changing the uibRoot folder.

  Updated the front page with links and explanations of the different sections.

New

• In the technical documentation, you can now access and search the main README as well as the current and archive changelogs (v1 & v2) in addition to everything else.

  Don't forget that you can access the tech docs on the Internet from GitHub AND locally from within Node-RED.

• nodes/web.js - Added web.isConfigured to allow a check to see whether web.setup has been called.

• nodes/sockets.js - Added socket.isConfigured to allow a check to see whether socket.setup has been called.

• Add a new icon to the main readme that allows editing of uibuilder code using VSCode either via a remote repository or via a Docker container.

Fixed

• Node-RED edge-case for credentials was causing node to be marked as changed whenever "Done" button pressed even if no changes made. Turns out to be an issue if you don't give a password-type credential an actual value (e.g. leave it blank). Gave the JWTsecret a default value even when it isn't really needed.
• Instance details page - CSS now loads correctly even if using a customer server port. Some Socket.IO details that were missing now returned.
• web.js - specifying a custom server port caused uibuilder to crash. Now fixed.
• Lots of tidying up of log messages, especially TRACE level.
• Accidentally include a node.js v14+ issue, now removed.
• Additional try/catch blocks to force better reporting if there is an error in the uibuilder module files.

Features, Bug Fixes and Code Refactoring

New

• Add new pre-defined msg from Node-RED that will cause the front-end client (browser) to reload.
• Add auto-reload flag to file editor - if set, any connected clients will automatically reload when a file is saved. (Only from the file editor in Node-RED for now, later I'll extend this to work if you are editing files using external editors).
• Add new function to uibuilderfe.js - uibuilder.clearEventListeners() - Will forcably clear any onChange event listeners that have been created. Partial update for Issue #134.
• Added initial documentation for front-end build tooling to technical documentation (general info and Snowpack).

Fixed

• Issue #126 - Security not turning on even if TLS is used.
• Update security.js template to remove simple false return if authentication fails - this is no longer valid.

Please note that the security features in uibuilder are still not complete and are not well tested. They also have some debug output. Other items on the backlog are currently taking preference.

Updated

• Bump dependencies to latest

• Add collapsible summaries to README.md

• Various updates to technical documentation

• Update chkAuth validation function to make it more robust

• Improve auth process logging and msg._auth.info checks

• Remove simple true/false return from auth processing as this is no longer valid

• uibuilderfe

  • Added check for uibuilder.start() having already been called and prevent it being run more than once. Partial update for Issue #134.
  • Add new function uibuilder.clearEventListeners() - see details in New above.
  • Added initial code for a simple alert - not yet ready for use.

• Internal code refactoring

  • Prep for adding the ability for uibuilder to use its own independent ExpressJS server

  • Rename uibuilder.js's nodeGo() function to nodeInstance() for clarity

  • Add dumpReq() to tilib.js - returns the important bits of an ExpressJS REQ object

  • Begin to add Node-RED type definitions

  • Add ExpressJS type definitions

  • Other linting improvements

  • The refactoring has removed several hundred lines of code from the main js file and
    simplified quite a few function calls.

  • Moved Socket.IO processing to its own Singleton class module.

    This means that any Node-RED related module can potentially require the socket.js module and get
    access to the list of Socket.IO namespace's for all uibuilder node instances. All you need is the uibuilder URL name.

    It also means that any module can send messages to connected front-end clients simply by referencing the module and knowing
    the url.

    Note that this currently only works once the class has been instantiated and a setup method called.
    That requires a number of objects to be passed to it. This happens when you have added and deployed a uibuilder
    node to your flows.

    But it does mean that, in theory at least, you could now write another custom node that could make use of the uibuilder communications
    channel. Of course, it also opens the way for new nodes to be added to uibuilder. However, a slight caveat to that would be that
    loading order would be important and you really must deploy uibuilder before any other node that might want to use the module.

  • Started moving ExpressJS web server handling to its own Singleton class module

    Again, this will mean that any module running in Node-RED could potentially tie into the module
    and be able to access/influence uibuilders web server capability.

    Works similarly to the Socket.IO class above. So it has to be initialised using a number of properties
    from the core uibuilder node.

    Currently, only the core ExpressJS app and server references are handled by the class. More work
    is required to move other processing into it.

• Include PR #131 - add Socket.IO CORS support

Feature Release

New

• You can now choose between front-end templates.

  Vue/Bootstrap-vue is still the default.

  Expand the "Advanced" settings to see the new dropdown. Note that uibuilder never overwrites your files so you either have to change the selection before the first deployment of the node or you have to delete the index.(html|js|css) and README.md files before changing the selection.

  Three templates are currently included, more may be added later:

  • An updated version of the existing default template that uses VueJS and bootstrap-vue. Contains an additional button demonstrating the new simple eventSend function.
  • A new "Blank" template. This does not contain any front-end libraries or frameworks. It uses just the uibuilderfe library with raw DOM commands.
  • A simplified Vue template. Contains the bare minimum to get you going.

  Templates are also now more comprehensive and flexible and contain README files for information.

  Templates will also warn you if you are missing a library that they depend on. Install them through the uibuilder library manager.

• The Editor will now tell you if you have missing dependencies for your chosen template.

  

  Useful for people who forget to install vue and bootstrap-vue now that they have been removed from the default install.

• When changing an existing node's URL:

  • The existing source folder is renamed

    No more losing track of existing code!

  • Folders as well as instances are checked for duplicates

  • You are now warned to redeploy straight away, before doing anything else

• When deleting a uibuilder instance, you are offered the chance to delete the source folder

• In the uibuilderfe front-end library:

  • Added a new public method: eventSend. You can use this to attach to any HTML DOM event (e.g. a button click).
    It will automatically send a msg back to Node-RED with details of the event.

    Details on how to use this are contained in the technical docs in the uibuilderfe-js page.
    You can access these docs directly in Node-RED either using the button in the configuration panel or the link
    in the help panel.

    The updated default template also contains an example button that uses the new feature.

    Note that you can use more than just button clicks. It will work with any DOM event that you attach it to.

Changed

• Better warning if you set/change a URL to one that already exists.

• When changing URL:

  • The original folder (if it exists) will be renamed
  • The uibuilder instance folders are also checked. The change is rejected if the folder exists.
  • You are warned that you need to redeploy before doing anything else.

  NOTE: You may have lots of old uibuilder folders lying around. If your url change is rejected and you can't think why, check the folders.

• Check for duplicate url moved to v3 Admin API. API Test file updated.

• Further improvements to the techical documentation. This is now available online as well as from the uibuilder node configuration panel and the help panel in the Editor.

• Improved links from the Node-RED Editor's help panel, particularly on how to use the uibuilderfe front-end library.

• Extensive improvements to the
  documentation for working with the uibuilderfe library in your front-end code.

• The default Vue template now defines the data section as a method instead of an object. This is recommended and prepares for Vue v3.

Bug fixes & Enhancements

Fixed

• Issue #106 Editor: When editing files, a filename with a leading dot did not set the filetype correctly.

• Issue #105 Editor: Attempting to edit a hidden file (with a leading dot) resulted in an error and white screen.

New

• Issue #108 You can now view the uibuilder package docs (the ones in this package) by going to the url <node-red-editor-url>/uibuilder/techdocs.

  The package docs use Docsify for formatting. The docs include a search feature as well.

  The docs are linked to from both the uibuilder help information panel and from a new button in the configuration panel.

• The config editor has a new button Instance Details. clicking the button will show a new page in a new tab. The page contains debug details of the exact settings for the uibuilder instance. This should help people better understand all of the settings including folders and urls.

Changed

Editor, "Edit Source Files" improvements:

• ALL folders and files within the <uibRoot>/<url> folder can now be edited.

• Soft- or Hard-linked folders and files can now be used. This lets you put your front-end resources wherever you like as long as you create a soft or hard link into the <uibRoot>/<url> folder.

• Added better information toasts on file create/delete actions.

  Pop-up notifications are now given when you create/delete folders and files.

• Made keyboard enter button do the default action in the create dialog windows.

• Added more information to the create/delete dialog windows. (url, folder name, file name)

• Issue #102 Relaxed the file-type checks when editing files. Allows for use of more ACE file-types and prepares the way for the introduction of the Monaco editor in Node-RED v2.

• Issue #107 Allowed the selection of any folder or sub-folders in the file editor.

  The editor still constrains you to the folder for the instance but any folder within that root can be viewed. New sub-folders can be created and existing ones deleted.

• Issue #109 Persist the selection of folder and file when editing.

  This means that closing and reopening the editor will return to the last edited file.

  Uses browser local storage and so does not work with Internet Explorer (which hasn't been supported by uibuilder since v3.0.0).

• Improved display when no file is available to edit or if the file cannot be opened.

• Started moving to new v3 admin API's that are more consistent with less overheads.

• Changed "Edit Source Files" button to say "Edit Files". Recognising the additional capabilities.

• Changed button link names in the configuration panel to clarify and accommodate the 2 extra buttons for the instance details and technical docs links.

uibuilder.js:

• Started to simplify and rationalise API checks and reporting. Deprecated /uibfiles, /uibnewfile, /uibdeletefile API's, replaced with new v3 admin API /uibuilder/admin/:url. Simplifies the admin API's, makes them more consistent and reduces the number of URL's.
• Added v3 admin API's to create new and delete files and folders
• Added /uibuilder/instance/<url> admin API. Is created for each instance. Calling it will show a detailed information page for the given uibuilder instance.

Other

• Updated dependencies
• Installer: Improved the post-install console message (Post Install takes a while). Also forces VueJS to v2.x (not v3 as yet which will soon be the latest version because there are currently too many breaking changes).