In recent years, AR and VR concepts and related designs have got overwhelmingly popularity and attention, and many technology companies have invested in this wave.
In the first half of 2017, I participated in the designing process of “Little White Wall”, a mobile app which is based on video-through logo recognition and AR
model contents including virtual display windows, purchase tools, and KOL showcases for different retail brands (H&M, Zara, MUJI, etc.).
Challenges and Solutions:
Challenge NO.1. Information overload and levels of brand contents:
Because the display window, in reality, carries many different levels of information such as outfit samples, prices, discounts and when we tried to integrate more of them of online comments, rates, and links, we got tons of confusing information awaited to be categorized.
So, we reorganized all of the brand contents on many levels: brand cards, brand content display pages, single-item buyer shows, and more.
Based on the contents needed to be presented at each step-in user’s exploration process and the relationship between the steps, we began to define the way in which AR content should be carried for each individual step. Instead of using 2D stickers, we chose a simple rectangular 3D model with a thickness to carry text and image content and named it “Block”. By configuring the “Blocks”, we can form a “Wall” in aligning with different length, shapes, and formats of contents. This solution helped us to project pages composed of inerratic cards that were both easy to read in real places and to keep integrity and consistency.
Challenge NO.2. The over-bended visual surface on the mobile phone
In terms of camera lenses, although bend surfaces were more vivid and realistic than the tiled ones, a camera-centric design would make surfaces too curved.
Solution-The setting and configuration of “Blocks”
We defined a more compromised and calculation-favored radius of the curved surface: we located the camera at the 1/2 of the entire radius.
The front lens of a mobile phone camera uses 35mm lens, so the field angle of the lens is about 60°. And the mainstream screen size is 16:9, so we calculated the Xfov to 36° by simple geometric calculation.
In order to show a more comfortable Block wall area, we hoped to carry 3 Blocks horizontally in the screen, so we divided the angle 36° into 3 pieces of angle 12° and positioned each piece of Blocks on the wall in the 12° area.
Then we designed a 1:1 square Block as the reference Block which can be combined into 1×2, 2×2, 2×3, 3×3 larger sizes according to the content needs. In order to avoid the possibility that the maximum Block would exceed the screen so that the users have to move the phone to read contents on a single Block, we limited Blocks to be 3 so as the wall could be composed up to 5×12.
And for Blocks, we defined functional Blocks and Content Blocks. Functional Blocks were equivalent to functional buttons, while content Blocks carried various types of information.
Challenge №3: Intuitive interaction in AR world
It’s long been a problem to make AR apps interact more intuitively with users, most of the apps were still using slide and tap, while the eye-tracking interaction was too expensive and unmatured to build.
Solution: Use the camera to simulate human eyes
This time, we tried something different: using camera rotating, pointing and staying to simulate the movement of a user’s eyes.
We defined the top-level control buttons that return and close the two buttons as AR content and placed them at the bottom of the screen for easy operation. The concept was similar to the basic operation of the web browser.
Key Elements for ONE-HANDED AR Interaction Design
Left Draw, Tap, Hover, Hold Still
Left Draw — Left Slide:
The way of the brand card flipping into the brand content wall was a little bit special because we wanted to realize rich visual experience when users flip the cards.
Tap — Click:
Basic interaction, finger click operation.
Hover — Hover:
In the AR use scenarios, the user needs to raise the mobile phone to view the content right in the front or above. This action has already caused a certain fatigue, and it would even increase the burden when the user needs to interact with the contents outside the range of the thumb when raising another hand to tap the screen.
The contents in AR scenarios were fixed. Users can read the contents through the camera by moving the mobile phone. Then we paralleled the interactive scenario to the one when users browse web pages on the computer screen and defined the center of the mobile phone screen as the position of the mouse cursor. Thus, users move the mobile phone as they do with the mouse cursor on the computer screen to view the AR contents, and the contents touched by the “cursor” would give a certain feedback.
In addition to using the mobile phone camera as human eyes to browse contents, this method is accompanied by a selection tool that provides the basis for the design of “Hold Still”.
After Hover interaction mode, we added Hold Still function to the targets that were aimed by the camera center (the “Cursor’), which would activate “click” function by hovering 0.5s on the same target.
The combination of Hover and Hold Still enabled one-handed operation of the mobile phone to browse AR content and perform a series of interactions.
Through the overall interaction process of AR content, we add a series of animations for each type of switching to make the entire “browser” page more smoothly.
The AR content appears by scanning the brand logo through the camera. After the scan is successful, the brand card appears first.
Card to Wall:
As mentioned before, the way to enter the brand content wall from the brand card is to flip the card through the left stroke. We use the flip effect to connect the process. After flipping the card, increase the visual impact of the wall.
Wall to Wall:
The content wall is made up of Blocks, which can be accessed into a new content wall by clicking on the jumpable Blocks. We define it as a secondary page and a three-level page.
At first, Wall to Wall’s switching effect was also expressed by rotating the entire wall, but whether it is the interaction (Click / Hold Still) or large wall rotation effect, it is not reasonable and comfortable. Therefore, the transition effect of Wall to Wall is designed to express the effect of the lower renderings to express the entry of the next level of the page.
In a content wall (page), when there is too much content, we define it as pageable. These content levels are all at the same level, so the dynamic effect of turning pages is designed to switch between left and right.
The content Blocks in the content-wall can be opened to show more information.
The concept of closure here is the closure of the entire AR content, which is equivalent to closing the entire webpage. In contrast to Card Emerge, the closing in the opposite way disappears.