UBIK HoloLens 2 Client Basics

The Hololens 2 Client's login menu offers a selective list of all locally stored profiles. On start up the last used profile is selected by default. After selecting the desired profile, pressing the login button starts the login process and disables the buttons of the login menu for the duration of the process. To interact with the login menu the finger pointer is used.

When logged in the user can use the system menu to access general features. The System menu is acessed by looking at the palm of your hand and consists of a grid of buttons.

Creation and Deletion

POIs are the HoloLens representations of UBIK content objects. They can be placed using the Create POI button in the System Menu. This opens the search menu, which allows the input of a search term using the Hololens 2 virtual keyboard. After the input is finished the search starts. When finished the result list below the search input is populated with results. Each result is a button which spawns the corresponding POI and closes the search menu. Upon creation the placement of the POI automatically starts. Using the Hand Ray allows you to aim the POI and Finger Tapping places it at that position. To delete a POI select it and in the System Menu press the Delete POI button, or press the Clear POIs button to delete all POIs. The position of POIs is persisted locally between sessions.

Levels of Detail

A POI can display more or less information based on the distance to the POI and it's selection status. Level 1 shows only the POI's position and its status color. Level 2 additionally shows the name, a short description, and the Icon. Level 3 additionally shows the high priority properties and if available the child count as well as the document count.

Manipulation

POIs can be moved at a distance by aiming the Hand Ray at the POI and holding the Finger Tap, or up close by directly moving the hand into the POI and holding the Finger Tap.

Context Menu

The context menu gives access to POI specific actions as well as detailed data about the POI object such as a full list of properties documents and children connected to the POI object.

Documents behave similar to POIs in the way they are placed and moved. To start placing a document press the corresponding button in the document list inside the context menu of a POI object. The placed document is displayed in a scalable window which also include a close button to delete the POI. Specific kinds of documents like videos and PDFs contain specific controls like play and pause for videos.

The Quest Log is a panel that can hold a list of tasks and work packages for the user. It can be opened from the system menu and is located centrally below the users natural field of view. By pressing the work package items in the list users can navigate into the work package and see its content. By default the Quest Log follows the user and stays in its dedicated position, but it can also be placed by the user manually.

Tasks are visually represented similarly to POIs, however the functionality is different. A Task gives feedback about its completion status, its value, and its position. Users can also interact with a task to change its value and completion state. Additionally a Task holds information about what needs to be done to complete it. This can be done in the Form of documents (Image, video, PDF) and text. To place a Task, users can select a Task from the Quest Log and use the menu button to open the context menu of the Task. Tasks that have already been placed are only visible when the corresponding work package is active in the Quest Log. For sequential tasks the visibility is limited to tasks that are not locked to allow focus on the active tasks. Task POIs also contain a property area to display its priority properties that contain a value. If a task contains important properties, a button to show/hide that area will show up. The property area displays one priority property at a time and with a previous and next button it is possible to slide through them. Another feature is that critical tasks are supported. They differentiate from non-critical tasks via a highlighted POI circle. Critical tasks can be configured on the server via the boolean property CRITICALITY.

Capture Image

POIs may allow adding image documents by taking a picture and directly adding it as a document. This is done via the Capture Media Menu that can be accessed via the system menu. When a POI is selected which supports creation of documents the " Take Photo" button in the System Menu is active and allows opening the Capture Media Menu. In the menu a "camera" button allows taking a picture which is then displayed as a preview for the user to discard or save.

Capture Video

As with capturing images, capturing videos is also done via the Capture Media Menu. When a POI is selected which supports creation of documents the " Capture Media Menu" button in the System Menu is active and allows opening the Capture Media Menu. In the menu a "record" button allows capturing a video. The record button changes into a stop recording button which ends the recording. The captured video is then displayed as a preview for the user to discard or save.

In the property list of the context menu of any POI it is possible to click on a property and edit it, if it is not locked and of a supported type. The property editor consists of the property name on top, an input area based on the current property type which allows the user to input information, and the cancle and accept buttons. Cancle discards any given input and accept saves and commits the input if it is valid.