ELFA XRForge

in progress:

• currently refactoring discrete expense-tracker example into raw-API example (described in README.md.old

Technical design

Index:

1. Synopsis
2. Architecture Overview
3. 3D DiD Primer
4.   ╰ markup language: easiest
5. NextGraph EscapeHatches

Synopsis

The following tries to breakdown the envision virtual worlds aspect, widget and app for ELFA suite.
The working title for the app is 'XRForge' but can be changed, dropped and implemented inside the widget if needed.

NOTE: Many of the content below is discussed already in this brainstorm document by Niko & Leon.
It is strongly adviced to read this as well, including the comments in the sidebar.

Architecture overview

This repository is basically the NextGraph discrete example (see README.md.old) of which the ORM-code should be removed. The goal is to use the RAW API's only, which allows for easy (JML world) XML-synchronisation between NextGraph Participants.

It was discussed that this approach is significantly faster compared to starting a Svelte/NextGraph from scratch.

                                                                             
              ┌────────────────────────────────────┐                         
              │ VR HEADSET | SMARTPHONE | DESKTOP  │                         
              └──────────────┬─────────────────────┘                         
                             │                        
          webbrowser app ────┴───────────────────
                    │                            
              ┌─────┴
              ▼
  ┌─────────────────────────────────────────┐             ┌────────────────────────────┐       
  | ELFA APP (XRFORGE)                      |             | Other ELFA APPs            |         
  |-----------------------------------------|             |                            |
  | editor (THREE editor or Janusweb)       |             |  include as viewerwidget   |
  |-----------------------------------------|        -----|  open .gltf in drive e.g.  |
  | jslib: WebXR viewer (THREE.js/Janusweb) | <-----'     └────────────────────────────┘
  | jslib: XR Fragment URL-parser           | 
  | jslib: JML parser / JanusWeb      <--------- XML ----   NextGraph RAW API's
  │                                         │           |  (see discrete example without ORM)
  └────  schemes plugins ───────────────────┘           |
               │                                        |  This is to collaborate on JML
               ├── http        ┌─did library─┐          |  in realtime with other NextGraph participants
               │               |             |          |
               └── did ◄─── fetch ───┬── stream <--------
                                     ▲           
        . . . . . . . . . . . . . . . . . . . . . . . . . . . 
                                     │           
                     scene1.jml      │                                   
                       | clouds.glb  │   world5.jml                     DiD's to JML/3D/etc files 
                       | chart.glb   │     | costs.glb                              
                       ╰ world5.jml  │     | turbine.gltf               A JML (world) is just an XML
                                     │     ╰ spreadhseet.xsl            which references other 
                                     │                                  DiD's and is rendered in 3D
                                     │                                   
        . . . . . . . . . . . . . . . . . . . . . . . . . . . 
                                     │                                   
                                     │                                   
                                     ▼                                       
                       ┌─────────────────────────┐                           
                       │                         │                           
                       │   nextgraph             │                           
                       │                         │                           
                       └─────────────────────────┘                           
                                                                                   

3D DiD Primer

how to hyperlink together 3D worlds based on DiD's pointing to 3D files/models.

3D objects can contain links to other 3D objects thanks href-metadata (similar to HTML) via the XR Fragments deeplinking spec.
This allows us to design a room/space/worlds connected via (hyperlinked) portals via markdown or 3D assets, or by embedding them in 3D files:

Markup language: easiest

This is the easiest and flexible way to link together virtual worlds via 3D Assets:

<!-- myscene.jml -->
<room>
    <object id="did:example:e8276ef827e68f72f6e872fe" pos='0 0 0' js_id="floor"/>
    <object id="did:example:2983982f793827f9837f298f" pos='1 0 0' js_id="building"/>
    <text value="world 1" pos='0 2 0'/>
    <link url="did:example:28e6f8276ef872f6e82876" pos='0 0 0'/> <!-- myscene2.jml -->
    <link url="did:example:2346f8276ef872f6e82876" pos='0 0 0'/> <!-- myscene3.jml -->
    <link url="did:example:777655676ef872f6e82876" pos='0 0 0'/> <!-- myscene4.jml -->
    <link url="did:example:aa232442366372f6e82876" pos='0 0 0'/> <!-- myscene5.jml -->
</room>

Above demonstrates JML (Janus Markup Language): it renders 2x 3D-files via DiD, and shows portals (<link>) to other JML-files.
JML is THE adviced unbloated framework-agnostic markup language for describing 3D scenes, a fully-interoperable scene-format also used in ecosystems like JanusXR and XRForge.

Markup language is highly suggested to collaborate in NextGraph, because:

• Enduser does not need (learning) special 3D editors
• NextGraph is optimized for realtime synchronization of (un)structured data

An widely used JML-viewer is JanusWeb which could be integrated into this Svelte app.
However, JML's very simply XML-parser-function could also be used instead any THREE.js too, in case janusweb is too much. The same goes for handling XR Fragment URL's in THREE.js.

NOTE: strong advice against AFRAME/R3F/X3DOM/etc markup: their markup language is hardcoupled/bundled to the viewercode, and cannot be hosted seperately. Instead, leveraging DiDs enables finer-grained, end-to-end encrypted management of 3D objects and JML-scenes (including password-protected portals). JML allows us to avoid monolithic 3D architectures without the complexity of X3D/VRML or javascript-bundlers.

Embedding DiD's in 3D Assets

Applications can also embed/export DiD's into 3D files (like glTF).
But also manually by launching Blender, to set up clickable portals (to other DiD's):

• add a textured plane (perhaps with a text or image)
• add the DiD as href custom property (href: did:example:fec2fe3cf2e3cfe2f3c2ec3f2ec3fe2 e.g)

For more detailed info see xrfragment.org#href

JanusWeb parses these href-values automtically, which boils down to somethink like this THREE.js code:

scene.traverse( (n) => {
  if( n.userData.href ){
    n.addEventListener('click', () => scene.teleport(n.userData.href) )
  }
})

For more info see https://xrfragment.org/#THREE.js

NextGraph Escapehatches:

For now webtechnology + tauri is used, but what if it's not enough for VR/AR?

• nextgraph is written in rust, so rust always possible
  • python possible
  • ffi possible
• tauri is safe (covers almost all platforms)
• if devices don't support WebXR then something like Xurfer can be used as OpenXR startingpoint
• Niko mentioned that publishing virtual worlds (hyperlinked JML and/or 3D fiels) to www is possible (via a nextgraph broker-app (requiring E2EE user consent))