|
|
Bring photos and videos to life with the anaglyph 3D effect seen here on the right - just put on a pair of 3D glasses! The anaglyph was generated with depth prediction, after processing the image with an edge detection filter.
Anaglyph 3D, an iOS and macOS app, creates anaglyphs for photos, Live Photos and videos by generating stereo image pairs from depth prediction.
Two photos taken with a slight horizontal offset—mimicking human vision—form a stereo image pair. When viewed with special glasses, Virtual Reality headsets, or free-viewing techniques like cross-eye or parallel viewing, this offset allows the brain to perceive depth, creating a 3D effect.
An anaglyph combines the left and right images of a stereo pair into a single image, creating a 3D effect when viewed with red-cyan glasses. Specifically, the left image is represented in shades of red, while the right image appears in shades of cyan. The color-matching lenses ensure that each eye receives the correct image, enabling depth perception.
Traditionally, a stereo pair is captured with a dual lens or double shot camera, as in our app Cameranaglyph. This app offers an alternative approach, using a single image to generate a stereo pair from depth prediction and machine learning.
|
|
An image and its anaglyph.
Each of your eyes sees a slightly different view because they are spaced apart horizontally. This causes objects to appear slightly shifted between the left and right views.
The amount of horizontal shift between common objects in both views is called disparity. Closer objects have a larger disparity than those farther away. You can observe this effect by looking at an object with one eye closed, then switching to the other eye. The closer the object, the more it appears to shift between views. Your brain uses this difference to perceive depth and distance.
Anaglyphs inherently display the disparity between a pair of stereo images, as they combine the images using two high contrast, complementary colors: red• and cyan•. The complementary nature of these colors is evident in that they are located at opposite corners of the color cube.
Every color can be decomposed into the primary colors, a triplet of values (R,G,B), with each value in the unit interval [0,1].
|
Red and cyan are complementary colors, positioned on opposite corners of the 3D color cube, or RGB color space, formed by all possible RGB color components.
The color cube in the RGB model can be mathematically described as the Cartesian product:
| [0,1] × [0,1] × [0,1] |
where each coordinate represents the intensity of the Red (R), Green (G), and Blue (B) components, respectively. This means:
The cube's vertices correspond to the eight extreme points where each component is either 0 or 1 (black, white, red, green, blue, cyan, magenta, yellow).
The edges correspond to smooth variations between two primary or secondary colors.
The interior points represent all possible colors that can be formed by mixing red, green, and blue in varying amounts.
Each color in the RGB space is a point in this unit cube, and the concept of complementary colors corresponds to opposite corners.
For example:
Red (1,0,0) and Cyan (0,1,1) are opposites.
Green (0,1,0) and Magenta (1,0,1) are opposites.
Blue (0,0,1) and Yellow (1,1,0) are opposites.
The set is completed by considering black and white in the RGB color cube.
Black (0,0,0) is the origin of the cube.
White (1,1,1) is the opposite corner (the farthest point from black).
Since black and white are opposites in this space, they can also be considered complementary in the sense that their sum is white:
| (0,0,0)+(1,1,1)=(1,1,1) |
The full set of complementary color pairs in the RGB color cube, where each pair sums to white (1,1,1), is:
|
View this image with a pair of 3D glasses, alternating between only the red lens and then only the cyan lens to see how each filter works. For example, when using only the cyan lens, the top word 'CYAN' becomes invisible against the now cyan-tinted background—only cyan light passes through the cyan lens.
3D glasses consist of a red lens for the left eye, and a cyan lens for the right eye. The color of an object is determined by which colors it absorbs. In the case of 3D glasses the red lens absorbs cyan, allowing only red to pass through — and that is why it is red. Similarly the cyan lens absorbs red, allowing only cyan to pass through — and that is why it is cyan.
An anaglyph is constructed using the red channel of the left image, and the cyan channel of the right image. When the anaglyph is viewed through the glasses the left eye only sees the red left image, and the right eye only sees the cyan right image. The brain then processes the left and right images, as it does for normal viewing, to give the illusion of depth.
Machine learning is a process where computers analyze data to make predictions without being explicitly programmed for each task. It employs a model to detect and analyze patterns within the data. Training involves optimizing the model by iteratively adjusting its parameters using labeled data to minimize prediction errors and improve performance.
One application of machine learning is depth prediction in images, where the model estimates how far objects are from the camera. By training on millions of images, the model learns to recognize visual patterns that indicate depth.
|
|
Images with their predicted depth maps and anaglyphs.
A depth map is an image that represents object distances by encoding them as grayscale pixel values. Using machine learning, it’s possible to predict depth from a single photo or video frame, creating a depth map. The depth map is used to create a disparity filter to generate a right and left image stereo pair by shifting pixels in the original image. The stereo pair can then be used to generate an anaglyph.
In machine learning, training means teaching a computer model by showing it lots of examples. The model learns patterns from these examples to make predictions on new, unseen data.
In the case of depth prediction on images, the goal is for the model to estimate how far away objects in a photo are from the camera. To do this, the model is trained on millions of images, each labeled with depth information (like how far objects are from the camera). By looking at these examples, the model learns to associate visual patterns with depth.
For example, in a set of images showing a street scene, the model might learn that objects closer to the camera (like a car) appear larger and sharper, while distant objects (like buildings) appear smaller and blurrier. The more images it sees, the better it gets at predicting depth in new images it hasn't seen before.
So, when you hear a model was "trained on millions of images," it means the model learned from a massive set of examples so it can now make accurate predictions on real-world data, like estimating depth in photos you take.
For more about how anaglyphs work, see the About section.
A spatial video differs from a conventional video in that each frame contains two images, a stereo pair — one for the left eye and one for the right eye — creating a sense of depth and three-dimensionality. Photos and Live Photos can also be spatial.
|
The difference between conventional and spatial video.
For spatial photos and videos that already contain stereo pairs, this app can generate anaglyphs directly from those embedded pairs.
Select a photo or video using the browser or from the sample media in the app menu. This app handles three media types—Images, Live Photos, and Videos—each of which is processed in its own specialized view to generate the anaglyph.
|
Use the browsers to load media from the Photos library or file system, and the app menu to try samples of each media type.
Photo: When you select a photo, the app opens the Image Anaglyph View to generate its depth map and anaglyph.
Live Photo: For Live Photos, the app will open the Live Photo Anaglyph View, which processes both the photo and video components.
Video: For videos, the Video Anaglyph View is used to generate an anaglyph, side-by-side or spatial video from the video’s frames.
Tap the App Menu in the top-right corner to access sample media and options, including how Live Photos are opened.
Since a Live Photo consists of a photo and a video combined, Live Photos can be opened as its photo or video alone, by selecting the option to do so in the sub-menu titled Load Live Photo as of the app menu:
|
Option to load a Live Photo as a regular photo or video.
The app menu provides media samples of all the supported types to familiarize yourself with how the app functions for each type, and provide material for discussion here.
|
Select a sample from the menu and it will be opened in the view appropriate for its type.
The Side-by-Side video is provided to discuss the Convert Side-by-Side to Spatial feature in the Video Anaglyph View.
Spatial media is provide to illustrate the dual ability of this app to generate anaglyphs with stereo pairs generated from depth prediction or by using the stereo pairs contained within spatial media.
Anaglyphs are generated using stereo image pairs using depth prediction.
If the media is spatial the app may can use the stereo pairs embedded within it to generate anaglyphs.
A stereo pair can also be loaded directly from the
app menu:
|
If the media is spatial, all of the media views will present an option to use the embedded stereo pairs, as follows for a spatial photo:
|
When a spatial image is loaded the left and right image of its embedded stereo pair is displayed.
|
|
Access sample media of all supported types in the app menu.
Tap the App Menu in the top-right corner to access sample media and options, including how Live Photos are opened.
Select a sample photo from the sample media in the app menu. The Image Anaglyph View will open with the selection.
|
|
|
Tap on the Generate Anaglyph to generate the anaglyph.
If you select an image, the Image Anaglyph View will open, where you can generate its depth map and anaglyph.
Swipe on the anaglyph image or tap the capsule-shaped button to toggle between the anaglyph and its depth map.
|
The Preview Disparity Intensity button opens a view to preview while varying values of this parameter.
The
Disparity Intensity parameter adjusts the pixel shift between the left and right stereo images generated from depth prediction. Increasing the value exaggerates depth differences, enhancing the 3D effect, while decreasing it reduces the disparity for a more subtle stereo effect — similar to adjusting contrast in an image.
Tap on the Preview Disparity Intensity button to experiment with varying values of the disparity intensity.
|
|
Anaglyphs with less and more disparity intensity.
|
Choose a preprocessing filter to make close objects stand out more.
This special effect option converts input images to grayscale or black + white prior to anaglyph generation to further enhance the disparity of the its red and cyan channels.
|
|
|
The grayscale and black + white anaglyph filter applied to the app's sample photo.
These filters can be applied to Live Photos and videos also.
|
|
Access sample media of all supported types in the app menu.
Tap the App Menu in the top-right corner to access sample media and options, including how Live Photos are opened.
Select a sample live photo from the sample media in the app menu. The Live Photo Anaglyph View will open with the selection.
|
|
|
Tap on the Generate Anaglyph to generate the anaglyph.
If you select a live photo, the Live Photo Anaglyph View will open, where you can generate its anaglyph. A Live Photo has an image and video component and an anaglyph version of each part will be generated. If you want to only generate the anaglyph for either the image or video then select that option in the App Menu.
Use the Preview Disparity Intensity to pick the best value of the Disparity Intensity. In the preview view you can select specific frames of the video to experiment with various values.
|
|
The Preview Disparity Intensity button opens a view to preview varying values of this parameter, on the photo and its video.
|
|
Access sample media of all supported types in the app menu.
|
All spatial media have the stereo pair source option.
As with all spatial media, spatial Live Photos have the stereo pair source option: to either generate stereo images from depth prediction or to use the stereo image pair it already contains.
Spatial Live Photos do not yet contain spatial videos. Therefore when generating an anaglyph using the stereo images stereo pair source, the video part will not be an anaglyph video, as it is when using stereo pair source depth prediction.
|
|
Access sample media of all supported types in the app menu.
Tap the App Menu in the top-right corner to access sample media and options, including how Live Photos are opened.
Select a sample video from the sample media in the app menu. The Video Anaglyph View will open with the selection.
|
|
|
Tap on the Generate Anaglyph to generate the anaglyph.
If you select a video, the Video Anaglyph View will open, where you can generate its anaglyph.
|
The Preview Disparity Intensity button opens a view to preview various values of this parameter.
The
Disparity Intensity parameter adjusts the pixel shift between the left and right stereo images generated from depth prediction. Increasing the value exaggerates depth differences, enhancing the 3D effect, while decreasing it reduces the disparity for a more subtle stereo effect — similar to adjusting contrast in an image.
|
Anaglyphs with less and more disparity intensity.
Tap on the Preview Disparity Intensity button to experiment with various values of the disparity intensity of any frame in the video.
|
The anaglyph video view display progress.
Anaglyph generation is accompanied by progress display. Generating a video anaglyph from depth prediction can be time consuming, depending heavily on the hardware support for it.
|
The difference between conventional and spatial video.
A spatial video differs from a conventional video in that each frame contains two images, a stereo pair — one for the left eye and one for the right eye — creating a sense of depth and three-dimensionality.
This difference is reflected in the options presented by the Video Anaglyph View.
|
The available video output types for stereo pair source depth prediction and non-spatial videos.
For a conventional video stereo pairs are generated from depth prediction on the single image extracted from a video frame. Each stereo pair is then used to generate three types of video output as follows:
Anaglyph: Each output video frame is an anaglyph created using a stereo image pair generated from depth prediction on the input video frame. The resulting video should then be viewed with red-cyan 3D glasses.
Side-by-Side: Each output video frame is a juxtaposition of the images using a stereo image pair generated from depth prediction on the input video frame. The placement of images is intended for parallel viewing. To see the 3D effect, the viewer must train their eyes to focus as if looking at a distant object, allowing each eye to align with the corresponding image.
Spatial: Each output video frame consists of a stereo image pair generated from depth prediction on the input video frame to generate a spatial video. A spatial video is a stereoscopic 3D video format introduced for the Apple Vision Pro.
The
Disparity Intensity parameter adjusts the pixel shift between the left and right stereo images generated from depth prediction. Increasing the value exaggerates depth differences, enhancing the 3D effect, while decreasing it reduces the disparity for a more subtle stereo effect — similar to adjusting contrast in an image.
Tap on the Preview Disparity Intensity button to experiment with various values of the disparity intensity for any frame in the video.
|
The available video output types when using stereo pair source stereo images on spatial videos. Since the input is spatial there is no need for spatial output type listed.
For spatial videos there is an option named Stereo Pair Source to pick between generating anaglyphs with the embedded stereo images that each frame contains, or from depth prediction on one of the images in the stereo pair - in other words treat the spatial video as a conventional video (as it is on older device models that do not support spatial videos.)
The spatial video options are labelled Depth Prediction and Stereo Images. Each mode is then used to generate three types of video output as follows:
Anaglyph: Each output video frame is an anaglyph created using a stereo image pair generated from depth prediction on one of the input stereo pair images.
Side-by-Side: Each output video frame is a juxtaposition of the images using a stereo image pair generated from depth prediction on one of the input stereo pair images.
Spatial: Each output spatial video frame consists of a stereo image pair generated from depth prediction on one of the input stereo pair images.
Anaglyph: Each output video frame is an anaglyph created using a stereo image pair extracted from the input spatial video frame.
Side-by-Side: Each output video frame is a juxtaposition of the images using a stereo image pair extracted from the input spatial video frame.
Stereo Images is the default stereo pair source for spatial videos. Note that this mode does not offer spatial video output type since the input is already spatial.
Video tools:
Trim: Trim the input video.
Remove Background: Remove the background from the input video.
Anaglyph Filter: Apply the current filter to the input video (if an anaglyph filter other than None has been selected below).
Side-by-Side To Spatial: Convert a side-by-side stereoscopic input video to a spatial video.
The input video can be trimmed from the current time to the end, or to the current time from the beginning of the video. |
Remove backgrounds to make the 3D pop effect more dramatic in generated anaglyphs. Note that if the video is spatial and the device supports it, background removal will generate a new spatial video, by removing the background in each image of each frames stereo pair. |
The video can be filtered using the current selection in the Anaglyph Filter Type below. |
If the input video is a side-by-side layout video, it can be converted into a spatial video, if writing spatial video is supported. For the more common parallel viewing mode, the left image should appear on the left side, and the right image on the right. For cross-eyed viewing, the order is reversed. A key clue is that foreground objects intended for the right eye appear shifted to the left relative to the same object intended for the left eye. Adjust the Parallel Viewing toggle accordingly. |
The player video can be trimmed from the current time to either the beginning or end of the video. If necessary apply this tool iteratively to isolate the desired range.
|
The video toolbar has buttons to focus attention on the left or right video player, as well as to control layout.
Tap on the left video select button to maximize the size of the player to facilitate selecting a trim time.
|
Select a trim point with the video progress bar (or seek bar) slider.
Tap on the player so its video seek bar can be used to select a time.
|
From the Trim Input menu select either To Here or From Here to trim the video from the selected time to either the beginning or end of the video, respectively.
Use Revert to revert back to the original video to start over if need be. You will be prompted to confirm you want to revert to the original.
Side-by-Side stereoscopic 3D videos, intended for VR headsets, can be converted to spatial videos. This is done by splitting each frame into two images for a stereo pair that is then stored in spatial video format.
|
Side-by-Side video sample in the app menu.
Try this by loading the Side-by-Side sample included in the app menu. Tap the App Menu in the top-right corner to access sample media, and select Side-by-Side Video.
|
Side-by-Side video open in the Video Anaglyph View.
In Video Tools select Convert Side-by-Side to Spatial to convert it to a spatial video.
|
Side-by-Side video converted to a spatial video in the Video Anaglyph View.
The spatial video conversion will automatically load, ready for further processing. Tap on Generate Anaglyph to generate an anaglyph video.
|
Side-by-Side video converted to an anaglyph video using its spatial conversion.
The anaglyph video was generated using the stereo pair source Stereo Images Put on your red-cyan filter glasses to see the 3D effect.
|
The export menu in each of the media views, Image, Live Photo and Video, resp.
The media generated by this app can be saved to the photo library, the file system including iCloud storage, and shared with other apps or sent via AirDrop. Each media view has an export menu for this purpose, as well as additional output features.
An example of additional output features: in the Live Photo Anaglyph view one of the export options is Spatial Live Photo From Depth Prediction. This generates a live photo in which the photo is actually a spatial image (i.e. stereo image pair) generated from depth prediction and the video is the original video (not a spatial video).
|
Name the export file and choose its destination.
When you choose to save to Files the usual system file navigation view will display so you can navigate to the desired destination and name the file as you like.
|
Confirmation saving to Photos library.
When you choose to save to Photos, upon success a confirmation dialog will appear.
Due to their special structure Live Photos can only be saved to the Photos library. However, the export menu can also save a Live Photo's photo and video separately, from which a live photo can be created (Which is how the Live Photo samples work.)
|
The system share view presented.
Sharing is useful for sending files to another nearby device via AirDrop.