T.E.A.M. research is based on experimentation. In order to both get a better understanding of the phenomena we are studying and to test the tools of our workflow, we felt even the need to confront existing projects of kinetic architecture. As a first case study we have chosen to deal with an iconic project of recent architectural history: the pavilion for the 1970 World Expo in Osaka by Maurizio Sacripanti. This architecture, unfortunately never realized, was presented at the competition of ideas held in 1968 to select the building that would represent Italy at the international exhibition.
1. Built with
“Osaka ‘70” is built using Unity as Game Engine. The reason for this choice was mainly for the implementation of ready-to-use VR technologies, which helped us in the early stage of the prototyping research phase. As the development process went further on, we were able to develop our custom Components and built a custom framework on the top of the Oculus’ one, especially for what concerned the hand tracking feature, which was yet experimental at the start time of the research project. Many tools we developed for the “Platform One” research project helped us to have a better view on what we would have done and how we would have implemented the tools needed to take Osaka ‘70 back to life: some of those tools were taken and brought in Osaka, and in fact Osaka ‘70 has many pieces in common with Platform One.
Another framework which helped us in the early stage, and that we took some elements from, is the open-source Mixed Reality Toolkit from Microsoft: the high customization of each Component allowed us to further change and adapt some scripts to our own necessities. A key role was determined by the lightweight yet customizable shaders of the MRTK: the feeling of something real yet visible only in the virtual world was achievable thanks to the custom shaders of the framework. They also helped us to keep pretty high performances, keeping in mind the limited resources available in the first generation of the Oculus Quest.
2. Getting started
Prerequisites
“Osaka ‘70” requires at least Unity 20219.4.28f to build and run. Android build support (both Android SDK and OpenJDK) module needs to be installed with Unity in order to build for the Oculus Quest platform (which is an Android-based device). Any other framework is already installed in the project.
Installation
- Download or clone the “Osaka70” repository.
- Open UnityHub launcher. Select the Project tab on the right side and select Open>Add project from disk.
- Select the folder called “Osaka70” containing the entire Unity project and open it.
- First opening should take a while since Unity has to build the local Library of assets cache, compiled shaders and so on.
- After the recompiling the project opens directly with LoadingScene scene as primary scene in the editor.
3. Usage
Customize the environment
Many aspects of the experience can be customized because many features have been developed as highly changeable:
- Some parameters can be changed directly in Unity, since some of them have been exposed as public in their related script and can be found under their script component
- Other parameters can be changed only in code, since they build up a more complex structure related to many different behaviors [each relevant script has its own documentation]
Build your package
Any running build packaged from Unity has to be exported as an Android Package (.apk). The project should be already properly set. Otherwise, check into Build Settings that:
- Android is selected as target platform. If not, select Android.
- Under Android settings on the right side of the panel, the Texture Compression must be set to ASTC value.
- If any setting has been changed, confirm changes selecting Switch Platform. A process of assets conversion starts. It should take some time depending on the hardware configuration of your computer.
If Android platform is properly set, you should be able to build your own package: open the Build Settings panel, select Build (or Build and Run if your Oculus is connected to your computer and set in the Oculus app), select your output saving location on your computer and wait for the built process to complete. After that, you can upload your .apk in your Oculus Quest using the Command Prompt (on Windows) or Terminal (on Mac) or apps such as SideQuest or the Oculus Developer Hub.
IMPORTANT: in order to upload your packages, Developer Mode for your Oculus Quest must be enabled!
4. Implementation
The Osaka experience is made up of many different components and specialized managers which handle every aspect of the experience. Since Osaka has a direct root in Platform One research project and takes some of the tools and components developed for that VE experience, some classes have been extended and replaced with a more specialized version of them, sometimes creating them newly from scratch, sometimes extending the existing ones, in order to work with a modified version of the environment. These classes are still implemented in the project because they are the building block for the final version, although they may not be used anymore.
In this section we are going to explore the general execution process of a running build of Osaka ‘70 in order to understand how the entire application works and what are the main components that play a central role in the key features of the experience.
As multiplayer framework we chose to use Normcore, a relatively new framework, highly extendable, XR compatible and well Unity-integrated. Normcore does not use RPC to keep clients up-to-date, instead each networked property and component stores its own data in a database that they call datastore, which is automatically kept in sync on all clients, sending only delta updates to save bandwidth. So, any custom Normcore component we needed to implement is constructed by two pieces:
- a RealtimeModel, which defines a set of data to store in a room datastore
- a RealtimeComponent, which derives from a RealtimeModel in order to modify data in the model and respond to real-time changes when they are detected
Normcore provides many built-in Components to quickly implement multiplayer in Unity games:
- Realtime: manages a Room object and synchronizes its datastore to RealtimeViews and RealtimeComponents in the scene.
- RealtimeAvatarManager: manages avatar, instantiating a custom Normcore avatar, with its own Realtime Components, for each client
- RealtimeView: a component that manages a set of RealtimeComponents on a game object.
We also needed to implement some tools and features that should have been kept in sync with all clients visiting Osaka. The first feature we wanted to include was the ability for a user to become the guide, the super user who could control other visitors and guide them through the experience. For that reason, we decided to provide some handy tools to the guide, and to the guide only, such as a system for navigation and teleport on pre-determined locations, the ability to call any waiting user for a more refined experience, and so on. Custom Normcore components that we implemented are:
- RealtimeNormcoreStatus component, which memorizes who is the current client designed as guide and allows him to call other clients from the waiting room to the main scene
- NavigationSync component, which detects changes to locations where the guide wants to teleport to and forces all connected clients to move where the guide desires
- RealtimeNormcoreSceneManager component, which detects changes to the current loaded scene and updates any counter which helps the guide to know how many users are in the experience. Since all the main Realtime components are kept in one GameObject, the NormcoreManager, which is persistent throughout the scenes, when a new scene is loaded every Manager has to properly reconnect its properties to the right object in the loaded scene: this component ensure that every Manager sets its properties as intended.
- NormcoreConnectionManager component, which handles the connection to the network and tries to reconnect to Normcore if the Oculus device disconnects from the Internet
- RealtimeNormcoreTourManager component, which coordinates the navigation of a large number of visitors (up to 15 clients in what we called the Guided Tour mode)
The Osaka experience starts its execution in the LoadingScene, a waiting room for clients. Here all Manager GameObjects are activated and start their processes:
- NormcoreManager: it coordinates every multiplayer aspect of the experience. It has many components, each one specialized in one aspect of networked layer of Osaka, from managing users connection to disrupts in the Internet connection.
- UIManager: it allows a user to show up the password panel. In order to become the guide for the current session, a client has to type the right password.
The guide privileges are granted if a client types the correct password. This system is handled by the Secret Menu object, which has a PasswordManager component where you can define the password (a 5-number string) and register for specific events (such as, when the correct password is submitted or the wrong one is typed). The secret menu is displayed only if the client knows where to interact: on the left side at the base of the little finger of the right hand there is a hidden collider that, when triggered by the index finder of the left hand, shows the menu to the user.
As soon as the password is correctly typed, the user is granted guide privileges and is taken directly into Osaka scene. Here, after some necessary elements of the environment are set and they start working in the background, any other feature is triggered by specific actions:
- GameManager: it coordinates with the TimeManager to control time inside the experience, to set properly the camera and the general execution of the application.
- TimeManager: it is structured by many other components, each of them related to a specific aspect of the environment, such as fog color, cloud opacity, skybox material… and it controls how these elements behave during time. TimeManager also activates the digital clock: it sets the light position and rotation to the current time, and then the day/night cycle is automatically animated. Day cycle is a bit faster than the real-time one, and night cycle is much faster.
- UIManager: it controls any interaction occurring to the buttons of the guide control panel, recording events fired by the user’s hands and calling methods on specific objects. Although any client has this manager, only the one which is the guide can access its functions. The UIManager also listens for teleport events and redirects them to the NavigationSync component of the NormcoreManager so that clients can be notified and be teleported where the guide wants to move. Teleport events can be directed to specific locations, fixed point in the map, or to the position of the guide, useful if visitors are roaming around the map.
- TourManager: it manages a large group of visitors, overriding the standard teleport locations with wider pre-fixed positions and other features (like a voice mode for the guide, called GodVoice, to not be spatialized and to be heard everywhere by each visitor)
- RotatorManager: it’s a local manager that ensures that the blades of the pavillion start rotating according to the original rotatory motion described by Sacripanti in his writings. Each blade moves independently from each other, but their timing is set by the timing of other blades. For each blade it stores starting time and rotatory period.
- FadeHelperSVG: a Bolt FlowMacro that manages the user teleportation when entering the portals. It requires Portal prefabs in the scene linked to PortalOut prefabs. In the Portal prefab there is a reference to the desired quote list variable to show a message during the transition.
5. License
Copyright (c) 2022 Poplab srl Permission is hereby granted, free of charge, to any person obtaining a copyof this software and associated documentation files (the “Software”), to dealin the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sellcopies of the Software, and to permit persons to whom the Software isfurnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in allcopies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS ORIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THEAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHERLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
6. Download
Here the link to GitHub. You can clone or fork the project, but we won’t accept any pull requests. Be free to explore the source and implement your own solutions:
For any information regarding the projects and their implementation please write us at team@poplab.cc