The game control page allows control of external applications from a remote device. Generally these are Unity/Unreal apps that are integrated with the Igloo Toolkit.
Usage
Click or tap the “Control game engines” button to begin controlling external applications.
You can control orientation, movement, and game controller button presses in a number of ways which are outlined below.
“Orientation” in an immersive environment is actually controlling the on-screen, world-space cursor. You’ll see this move on the ICE Canvas when you change your orientation through any of the control methods.
If you’re using a mouse and keyboard, your mouse will be locked to the Control Panel window, and all further mouse movement and keyboard interactions will be forwarded. Press [Escape] to unlock your mouse and stop controlling the ICE cursor.
Mouse movement will adjust your rotation within the scene. Left clicking will send an A button press.
Keystrokes will send movement messages or button presses. See below for a keymap of these controls.
If you’re using a touchscreen device with gyroscope support, and your device is in a portrait orientation, the page will try and use data from your device’s gyroscope to control your orientation.
You can control your movement with a virtual joystick. Tap + drag anywhere on the dotted area to start using this joystick.
Buttons are shown at the bottom of the screen.
It’s possible when controlling orientation with a gyroscope that you’ll be physically facing a different direction to the on-screen cursor.
To adjust for this, hold down the cube in the top right and rotate your device/body to point at the on-screen cursor. When you release the cube, you’ll begin controlling orientation from this new direction instead.
On iOS, you’ll be asked to approve use of the gyroscope when pressing the “Control game engines” button. If you deny this, the page will fall back to its no-gyroscope behaviour (described below).
If you’re using a touchscreen device without gyroscope support, you pressed the “Control game engines” button with your phone in a landscape orientation, or denied permission to use the gyroscope, the page will provide two areas for virtual joysticks - one for orientation and one for movement.
Tap + drag anywhere on the screen to start using a virtual joystick.
Joysticks on the left hand side of the screen will control movement. (Blue in this image)
Joysticks on the right hand side of the screen will control orientation. (Green in this image)
Buttons are shown in the top-right of the screen.
Keyboard controls
Keyboard/mouse input | Result |
|---|---|
W | Move forwards |
A | Move left |
S | Move backwards |
D | Move right |
Z | A button |
X | B button |
C | X button |
V | Y button |
Mouse movement | Change camera orientation |
Left click | A button |
DualSense controllers
ICE Control Panel also handles input from DualSense controllers, utilising the gyroscope for rotation.
Connect the DualSense controller to the device running ICE Control Panel via Bluetooth, then open the game engine tab and press one of the circle/square/cross/triangle buttons to ensure it’s connected. The “Start casting” button will show a PlayStation logo to demonstrate this. Press the PlayStation button to start casting.
The left stick provides movement control, and the DualSense’s gyroscope is used for orientation control.
Calibration of the orientation offset is available using the “mic mute” button on the DualSense controller (labelled M in the diagram above). Hold down this button and rotate to face the onscreen cursor, as you would using the touch controls described above.
Troubleshooting
Ensure ICE Control Panel is up to date.
Check the Unity app is accepting any input from the game engine tab (does the screen rotate when moving the mouse)
If the left stick is working, but the gyroscope isn’t, try updating the controller's firmware.
Control method selection
Pre-requisites
As for all ICE Control Panel usage, the Igloo Core Service must be installed and running on the media server.
The Igloo Controller Service also needs to be installed and running on the media server.
See Desktop UI - Setting up a Game Engine Layer for instructions on setting up an ICE layer to display the application, and correct configuration of that application (if using the Igloo Toolkit).
External applications that you want to control need to either be using the Igloo Toolkit (for Unity/Unreal), or accept the OSC messages outlined below.
OSC output
Messages to the Igloo Controller Service are sent on port 10000. Messages to external applications are sent on ports defined in the config file (by default 9007 and 9008).
Movement → Igloo Controller Service
/controller/0/leftThumb [x] [y]
x and y are the virtual joystick’s coordinates, either from an on-screen virtual joystick or calculated from WASD presses.
Results in software Xbox controller moving its left thumbstick
Orientation → External application
/ximu/euler [y] 0 [x]
x and y are components of a rotation Euler describing the user’s current world-space rotation.
This is initialised at (0, 0, 0) when the user presses the “Start controlling game engines” button, and reset whenever they stop casting.
Changes to the rotation are tracked by the page and the resulting world-space rotation is provided. This means that the game engine’s camera will jump back to a (0, 0, 0) rotation between control sessions.
The Igloo Toolkits for Unity and Unreal accept this OSC by default.
Buttons → Igloo Controller Service
/controller/0/[button] where button is one of [ 'buttonX', 'buttonY', 'buttonA', 'buttonB' ]
Results in software Xbox controller pressing the button.