Automatic Calibration and Real Time Alignment Guide
Last updated
Last updated
This guide will show you how to:
Perform an automatic calibration;
Activate the real time projector alignment feature;
Enable tracking for projection Screens.
Completing the steps listed above will allow you to use PHOTON’s auto-calibration feature. This step-by-step guide describes all the interface manipulations needed to achieve an auto-calibration. The set-up described in this guide is a simple one that was selected to facilitate comprehension. Your own set-up size, configuration and complexity might vary.
Once a video projection set-up has been automatically calibrated, it is possible to activate real time video projector alignment and screen tracking. Sections 1.3 and 1.4 cover these topics.
To perform an automatic calibration you will need to have a fully working set-up that includes video projectors and their screens, cameras connected to a POE network switch and at least one PHOTON server. The Mosaic Topology must be correctly configured PHOTON Servers
(the Mosaic Topology should be enabled on each server) before completing the next steps.
Creating the Server Group will be the first step of the Automatic Calibration Process. Make sure the network is correctly configured before starting the process.
Select {System > Settings} from the main menu or press [Alt+S] to open the Settings module.
Click on the network ring [pixbutton] button to open the Create Network Group window.
A list located on the left side of the window will allow you to include or exclude servers from the group by checking or not checking their associated checkboxes. If PHOTON has been launched on all of your servers, and if the servers reside on the same network, they should appear in the Photon Selection servers list located on the left side of the Settings window (while the PHOTON tab is selected).
Select the servers you want in the group by checking the boxes next to their name. When selected, the servers will be identified by a green check mark.
Click on the [Apply] button to create a Server Group.
Once a Server Group is created, you should make sure the PHOTON Network is fully functional and that all of your servers are communicating properly.
Select {System > Network Monitor} from the main menu or press [Alt+M] to open the Network Monitor module. This module displays information about each server in the Server Group.
PHOTON Servers operate differently depending on the selected System Mode. It is also crucial to specify which Server has control over the cameras. A PHOTON Group is organized around a primary server which maintains synchronization and control over the remaining secondary servers. This primary Server is usually a Controller Server while the latter are Display Servers. Display Master Servers act as in-betweens, meaning they share the properties of both Display and Controller Servers. ALBION Servers are dedicated to streaming tracking data. All of the following manipulations should be done from the Server acting as the primary Server.
Determine which Servers will play which role in the Group.
Select {System > Settings} from the main menu or press [Alt+S] to open the Settings module.
Select a Server from the Group by clicking on its entry in the Photon Selection list.
Choose an option from System Mode dropdown menu. The selected option will determine the Server’s System Mode.
Repeat steps 3 and 4 with each entry of the Photon Selection list.
Now that the System Modes have been attributed, determine which Server will take over the cameras during the automatic calibration process. The server will either be a Controller, an ALBION or a Display Master server depending on the configuration of your set-up.
Click on the [Control] tab located at the top of the Settings window.
Select the option corresponding to the name of the Server which will take over the cameras from both the ALBION Control dropdown menu and the Auto-Calib Control dropdown menu.
You will now create in PHOTON virtual video outputs that will be linked the physical video projectors. Make sure the video projectors are powered and connected to the Servers before completing the next steps.
Open the 3D Scene module by pressing [Alt+C] or by selecting {System> 3D Scene} from the main menu.
Select {System > Settings} from the main menu or press [Alt+S] to open the Settings module.
Select a Display Server by clicking on its entry from the Photon Selection list.
Click on the [+] button to add Virtual Video Outputs for the selected Server. Each Virtual Video Output must correspond to a Physical Video Output.
Each time a Virtual Video Output is created a Confirmation Box will open and ask if you want to create a Projector in the 3D Scene. Click on the [Yes] button each time.
A square-base pyramid corresponding to a newly created video output is simultaneously created in the 3D Scene as a primitive. This primitive is representation of the Virtual Video Output and is linked to a real-life video projector.
Repeat steps 3, 4 and 5 for every Display Server in the Group. You will now have to make sure that the physical video projectors are correctly connected and identified.
Navigate to the 3D Scene module and select the Calibration mode by clicking on the [Calibration] toggle. Once the Calibration mode is selected, the UI will turn red.
Click on the gear button to open the Options menu.
Enable [Display Ident] by clicking on its toggle. You should now see the projectors ID cards displayed in your set-up. Each projector will display an identification card that features a unique color. If a projector does not display an ID card, either it is not powered, connected or it is not linked to a Virtual Video output. Once you have verified the status of your projectors, click again on the [Display Ident] toggle to disable the function.
ALBION is a PHOTON module dedicated to tracking. Even if you do not plan to track anything, ALBION is solicited in the automatic calibration process. The module will gather position data and IDs generated by the Copernics (VYV’s optical marker system).
Open the 3D Scene module by pressing [Alt+C] or by selecting {System> 3D Scene} from the main menu.
Create a new instance of ALBION in the 3D Scene by selecting {Primitive > Automation > Albion}. You should now see a new entry in the Scene Primitives list named Albion. (By default, ALBION is enabled when created.)
Make sure that the Albion entry in the Scene Primitives list is selected (it should be highlighted). If not click on the Albion Scene Primitives list entry.
Press [P] or select {Options Panels > Properties} to open the Properties panel. (The displayed properties will be related to Albion since it was selected.)
Unfold the Calibration section of the Properties panel by clicking on the + sign.
Click on the [Auto Create Cameras] button. An Information Box will open and the display the IDs of the cameras that are connected to your system. If a camera seems to be missing, there might be a connection or network issue.
Close the Information Box by clicking on the [OK] button. Entries were created for the cameras in the Scene Primitives list.
Camera and projector calibration is based on correspondences established between the real world devices and their virtual representation in the 3D Scene module. The first step in establishing these correspondences is the creation of a Ground Volume reference. A Ground Volume is a virtual 3D object that you will match to a physical object through the use of Copernics.
Warning: There is many approaches to the creation of Ground Volume. Depending on the context where you plan to achieve an automatic calibration and tracking, some type of 3D objects might perform better than others. The technique involving a plane described in this step-by-step has been chosen for its simplicity. In contexts where projection surfaces and performers move in a tri-dimensional space, it is preferable to use a tri-dimensional shape to act as a Ground Volume. The points making up the shape should be well distributed along the space where the surfaces and performers will move. Instead of using a built-in Primitive (plane, cube, etc.) that you modify when creating the Ground Volume, you should use a total station to create a precise points cloud that will be imported in PHOTON. Doing otherwise (measuring positions without optical precision equipment and applying modifications manually to built-in primitives) will lead to errors that will compromise the calibration.
Install a physical rectangular surface on the stage fitted with four Copernics, each resting at its corner (the top of a road case or table could work). Make sure the Copernics are powered.
Measure the rectangular surface (length and width) in centimeters. If the surface is elevated (resting on a small table rather than the floor, for example), measure its height.
Add a virtual plane in the 3D Scene by selecting {Primitive > Default objects > Plane}.
[Double-click] on plane’s entry in the Scene Primitives list to open a text field. Rename the plane “Ground Volume”.
Press [T] or select {Options Panels > Transformations} to open the Transformations panel.
Use the R.x slider to rotate the plane so it matches the orientation of the physical surface. The rotation is expressed in degrees. [Double-click] on the slider to open a field where you will enter 90.
Use the S.x and S.y sliders to modify the virtual Ground Volume so its dimensions’ match with the physical one. Even if the Scale sliders are not dimension sliders (they act as scaling coefficients applied to the object’s original size); the easiest way to set new values for the virtual plane is to apply directly in the sliders the metric measurements made earlier (100 centimeter = 1.0) [Double-click] on the sliders to open fields where you will enter the metric values.
Each Copernic has a programmable ID number. Before continuing the automatic calibration process the next steps, you should know the ID numbers of the Copernics attached to the physical Ground Volume. Follow these steps if you need to discover their IDs:
Connect the Copernics to a USB port on the Controller server.
Open the I/O Manager module by pressing [Alt+I] or by selecting {System >I/O Manager} from PHOTON’s main menu.
Click on the [Copernics] tab.
Click on the refresh button to populate the Available Copernics list. You should now be able to see their IDs.
Now that the planes (virtual and physical) have matching orientation and dimensions, you will have to link the two structures by using the four Copernics located on the corners of the physical one.
In the 3D Scene module, switch to Operation mode by pressing 4 or by clicking on the [Operation] button. Once it is selected, the monitor screen will turn blue.
Click on the gear button to open the Options panel.
Pay attention to the Rigid Points section of the Options panel. The Rigid Points dropdown menu will allow you to select pre-declared Rigid Points and assign them ID numbers. This is how the correspondence will be created between the Copernics and the virtual Ground Volume.
[Double-click] on a corner of the virtual Ground Volume in the 3D Scene preview. The corner is now listed as a Rigid Point and is accessible as an option in the Rigid Points dropdown menu.
Select the Rigid Point from the Dropdown menu (it should be listed as Rigid Point #0).
Double-click on the ID text field located below the menu to change the current -1.00 value for the Copernic ID number.
Repeat steps 4 to 6 with the remaining corners.
Select the Albion entry from the Scene Primitives list by clicking on it.
Press [P] or select {Options Panels > Properties} to open the Properties panel.
Unfold the Calibration section of the Properties panel by clicking on the + sign.
Select the virtual Ground Volume in the Ground Volume Reference dropdown menu.
The Ground Volume will be used by ALBION to perform calibration for the cameras. The physical Ground Volume should be located on a stage section where it will be seen by all cameras.
Install additional calibration lines featuring Copernics in order add references for cameras and projectors during the subsequent calibration operations. Calibration lines should be well distributed on the stage space to provide enough data for PHOTON to be able reconstruct its volume.
Click on the [Calibrate Cameras] button in the Properties menu to launch the automatic calibration process for cameras.
A calibration report will be generated. Error per camera should be under 0.5. You will have to recalibrate if the error goes above this number. Reconsider the Copernics positioning and quantity or if the error goes above this target. The cameras’ orientation, focus or state (on/off) might be the source of the problem. Occlusions might also cause problems. Make sure there is no physical problems as those described above before restarting calibration.
Some cameras might be affected by lens distortion. To solve this matter follow these steps:
Select all cameras by clicking on their entry in the Scene Primitives list.
Press [P] or select {Options Panels > Properties} to open the Properties panel.
Unlock the Lens Distortion section of the Properties panel by clicking on the + sign.
Click on the Lens Distortion button.
Click on the [Calibrate Cameras] button again.
There are 2 types of cameras in an automatic calibration set-up. Some cameras are physically paired with projectors (they are attached to them and are aligned with their lens) while others are pointed at the stage area without being associated with a projector.
Before attempting the projector calibration, you need to specify explicitly to PHOTON which camera is tied to which projectors. Follow these steps to establish the pairs in the software:
Select the 3D Scene Operation mode by clicking on the [Operation] button. (This is done to change the color of the UI to facilitate camera identification.)
Select a camera by clicking on its entry in the Scene Primitives list. Notice that the 3D Scene representation (square pyramids) of the camera also gets highlighted. Since the cameras’ position and orientation have been established during the previous calibration step, you will be able to understand which camera in the list corresponds to which physical camera.
Double-click on the selected camera’s entry in the list to open a text field where you will enter a new camera name that will reflect its physical position (leftStageProjCam_01, for example).
Repeat steps 2 and 3 with each camera of the Scene Primitives list that is physically paired with a projector.
Reorder the cameras in the Scene Primitives list by click-dragging their entry to facilitate identification and subsequent manipulations.
Switch the 3D Scene to the Calibration mode by clicking on the [Calibration] button. The UI will turn red and the Calibration tools panel will open.
Select a projector by clicking on its entry in Scene Primitives list. Click on the target icon in the Calibration Tools panel. The projector will emit red light; this will help you identify the projector correctly.
Double-click on the selected projector’s entry in the list to open a text field where you will enter a new projector name that will reflect its physical position (leftStageProj_01, for example). The name should refer to its paired camera to facilitate identification.
Repeat steps 7 and 8 with each projector of the Scene Primitives.
Reorder the projectors in the Scene Primitives list by click-dragging their entry to facilitate identification and subsequent manipulations.
Once you have identified projector-camera pairs, you will specify parenting for each of them (camera = parent, projector = child). Ctrl-click-drag the projectors entries onto their matched cameras entries in the Scene Primitives list.
The reconstruction process will store the Copernics position data in an array so it can be retrieved later on. All of the Copernics on the stage should be seen in the 3D Scene representation before launching this process. Be wary of occlusions, reflections and daylight as they will create calibration errors.
Make sure the Calibration mode is selected (if not, click on the [Calibration] button.)
Select Albion from the Scene Primitives list by clicking on its entry.
Press [P] or select {Options Panels > Properties} to open the Properties panel.
Open the Tracking - Reconstruction section of the Properties panel by clicking on the + sign.
Click on the [Compute Reconstruction] button. An Information Box will open showing how many Copernics were used in the reconstruction process.
Click on the [OK] button to close the Information Box.
The projector calibration process relies on the projection of light patterns which are picked up by the calibrated cameras. Make sure the projectors are physically aligned with their paired cameras and are throwing light that can be seen by the cameras.
Initial calibration
The first attempt at calibration is done without any modifications to vision-related parameters. It is expected to be imperfect but offer a base on which to work.
Shift-click on the projectors’ entries in the Calibration Tools panel to multi-select them.
Click on the SLAC button to open the SLAC Options panel. (SLAC stands for Structured Light Automatic Calibration.)
Press the play button to calibrate the currently selected projectors.
Look at the camera view to check if the process is going well. Calibrated projectors will display a calibration error indicator. The displayed number should be below 0.5. Calibrated projectors that have an error over 0.5 have to be calibrated again.
Refined calibration
The second attempt at calibration should aim at improving the calibration of projectors who have a calibration error over 0.5. It is possible to select individual projectors and leave those that are well-calibrated untouched.
Open the SLAC Options panel by pressing on the SLAC button.
Open the Options panel section by clicking on the + sign.
Click on the [Show focus Grid] toggle to enable the focus grid.
Make sure the 3 dropdown menus located below the Focus Grid slider show the following selected options: {Sliding Patterns}, {Copernics}, {Rigid Points}.
Select the camera linked to the projector you wish to recalibrate by clicking on its entry in the Calibration Tools panel list.
Select {Apply Projector Settings} from the SLAC Settings dropdown menu. This option will display the camera’s point of view in the Calibration Tools preview while you modify visual parameters.
You should notice some reflective points in the camera’s point of view. The following manipulations are aimed at creating conditions in which the reflective points stand out from the background. You should also make sure there is enough reflective points in the camera’s point of view. Having less than 7 visible points will seriously hinder the calibration process.
Identify the projector you want to re-calibrate. Click on the eye icon located to the right of the other projectors’ name to blank them.
The two main parameters you will have to modify are: Light intensity, located in the SLAC Options panel: this parameter will control the global amount of light emitted by the projector during the calibration phase. Camera exposure, located in the camera view (while {Apply Projector Settings} is selected): this parameter will allow you to control the exposure of the camera paired to the projector being calibrated.
Once you think the visual context for a projector re-calibration is optimal (reflective points are standing out of the background), right-click on the projector’s entry on the Calibration Tools list to open a contextual menu and select {Auto-Calibrate (Full)}. Adjust the parameters and recalibrate until you reach an error below 0.5
Repeat steps 8,9 and 10 with each of the projectors that need to be calibrated again. Be careful, if you select “Apply Projector Settings” to multiple cameras at once without launching projector re-calibration, you will saturate your network with video feeds and lose connectivity.
In order to validate if your final calibration is acceptable, click on the gear icon to open the Options panel. Click to enable the [Enable] toggle and the [Display Cross] toggle. You will see cross markers projected on the reflective markers. The centers of the crosses should land on the reflective points. If you have difficulty seeing the crosses use the Line Width slider to enlarge them.
If you experience problems with the projectors calibration, you might be in a situation where the paired cameras are not seeing enough reflective points. You can move the calibration lines around so you give additional references to the system. Remember however that if you move the calibration lines, you need to click on Compute Reconstruction in the Albion properties before attempting any new camera and projector calibration.
Once cameras and projectors are calibrated, you will integrate the Screens used on your Stage. This will be done by adding 3D models that correspond to your real-life projection surfaces.
Adding Screens
Either your Screens have been previously modeled as .OBJ 3D objects or they are simple geometrical shapes such as planes and can be created directly within PHOTON.
.OBJ models
Open the File Manager module by pressing [Alt+F] of by selecting {Show > File Manager} from PHOTON’s main menu.
Click on the import files button located in the left-corner of the File Manager window. A Windows Explorer window will open allowing you to locate and import the .OBJ files that correspond to the screen you want to integrate. Select the files and click on the [Open] button.
Click on the refresh button in the File Manager window and select the [3D Object] entry amongst the Playlist filters. You should see the .OBJ files.
Close the File Manager Module and go back to thee 3D Scene module.
Select {Primitive > OBJ> yourscreenmodel.obj} from the 3D Scene main menu. The Screen will appear in the Scene Primitives list. You can now make this Screen visible to all or some of your projectors.
Built-in primitives (probably a plane)
Select {Primitive > Default Objects > Plane} from the 3D Scene main menu. The plane will appear in the Scene Primitives list.
[Double-click] on plane’s entry in the Scene Primitives list to open a text field. Rename the plane so you can identify it as a Screen.
Open the Transformations panel by selecting {Options Panels > Transformations}.
Modify the Primitive’s Scale sliders so their parameters reflect the Screen’s dimensions.
Making Screens visible for Projectors
Screens have to be made visible for projectors in order to display content. By specifying which projector’s sees which Screen, PHOTON limits the number of rendered video context it needs to process.
Enable the 3D Scene’s Calibration mode by clicking on the [Calibration] toggle.
Select the first Projector you want to link to a Screen from the Calibration Tools list by clicking on its entry.
Click on the lock toggle to avoid disrupting the projector’s calibration.
Drag and drop the Screen you want to be seen from the Scene Primitives list to the Calibration Tools preview. The Screen has now been made visible for that specific projector. Repeat this with all the Screens this projector should see.
Repeat steps 2,3 and 4 with all the projectors in the Calibration Tools list.
This feature allows the PHOTON to maintain calibration even if projectors are moved around. Once the PHOTON achieved an automatic calibration you can follow the steps to enable real time alignment. Keep in mind that a moving projector will keep its calibration only if its paired camera sees enough Copernics in its field of view.
Select the camera paired with the mobile video projector in the Scene Primitives list by clicking on its entry.
Press A or select {Options Panels > Automation] to open the Automation panel.
Open the RT Calib panel section by clicking on the + sign.
Select {Albion} in the Reference Object dropdown menu.
Click on the [Enable] button.
Select the same camera in the Calibration Tools list. You should see a “pos error: xx” value that keeps flickering.
Click on the lock button so you won’t be able to modify the calibration by clicking in the Calibration Tools preview window.
Before following the steps that will allow you to track a projection surface, you will have to fit the real world Screen with Copernics so the cameras linked to PHOTON will be able to see it. The following instructions have been designed for a plane fitted with 4 Copernics at its corners.
Select Screens you have integrated in the 3D Scene (and fitted with Copernics) from the Scene Primitives list by clicking on its entry.
Switch the 3D Scene to the Operation mode by clicking on the [Operation] toggle.
Click on the gear button to open the Options panel.
Double-click] on a corner of the Screen in the 3D Scene preview. The corner is now listed as a Rigid Point and is accessible as an option in the Rigid Points dropdown menu.
Select the Rigid Point from the Dropdown menu (it should be listed as Rigid Point #0).
Double-click on the ID text field located below the menu to change the current -1.00 value for the Copernic ID number.
Repeat steps X to X with the remaining corners.
Select the Screen in the Scene Primitives list by clicking on its entry.
Press A to open the Automation Window.
Click on the [Active] toggle located below Albion Options.