It's crucial to understand that we won't be creating the entire app in one go. The initial AR experience serves as a foundation for exploratory research, involving a significant number of participants using the AR prototype and sharing their insights. Their invaluable insights, gathered are the building blocks for a follow-up user-centered design approach to design and deploy the final product.
Background of the Clarissa Street History & Jazz
In the exploration of Clarissa Street's jazz heritage, the "Pythodd Club" emerges as a paramount venue where internationally acclaimed musicians graced the stage during the vibrant 1960s jazz era. This club stood as a beacon of musical brilliance, becoming an integral part of Clarissa Street's cultural identity. Our team meticulously assessed the ambiance of the Pythodd Club by engaging with government officials from the city of Rochester. Through these interactions, we gathered images and video recording of the Pythodd Club.
Additionally, we delved into the memories and experiences of the Jazz Club Members of Rochester to compile detailed playlists that showcased the renowned jazz tunes that once reverberated within the club's walls during the 1960s.
Contextual Inquiry to the Location
We visited the Pythodd Club location to comprehend the spatial layout, determining the placement of our 3D model and assessing the surrounding environment. During our visit, we identified real-world obstacles that might obstruct the 3D model in the AR space. Additionally, we conducted a safety audit for users, particularly considering the location's positioning at a busy intersection where awareness of moving traffic is crucial. Finally, we measured the available dimensions of the space to ensure our real-life-sized virtual building fits seamlessly within the environment.
Picking the Tools for Development
We used Unity's AR Foundation framework, enabling cross-platform development for both iOS and Android, integrating features from ARCore for Android and ARKit for iOS. This approach streamlined development, eliminating the need for separate coding for each platform and reducing overall development time. Also the device we used for testing are “iPhoneX, iPad, Asus ZenFone AR”
Location Based Tracking System using GPS
Initially, we had built an in-house location-based tracking algorithm for Unity AR to place 3D models in world coordinates. Simultaneously, we explored other outdoor location tracking Unity integrations. Each option had its pros and cons, but none met our team's requirements in terms of pricing and functionality flexibility.
Updating the Model
While experimenting with a generating label in the geo-location, our 3D modeling artist provided a draft version of the Pythodd Club. We subsequently replaced the labels with the draft version.
The solution faced challenges due to inconsistent GPS calibration across different devices, leading to varying precision. The virtual object frequently had an offset height in real-world space until the GPS stabilized. Moreover, under low-light conditions, especially at night, the real environment lacked detail, resulting in a less realistic visual experience in AR space.
Building the AR Interactions
At this point, we updated our model further and Rather than offering users a static model for research purposes, we opted to include interactive elements. This approach allows users to explore the prototype's flexibility freely and offer suggestions without constraints. We implemented ray-based interactions around specific objects in the model, like windows, doors, and entrances. Interacting with these objects provides users with media and music experiences
“The known issue at this point was fluctuating gps values for same location. My co-op at MAGIC Spell Studios concluded at this point (Jul 2022), leading to reduced involvement in the project. However, after the summer 2023, I rejoined the team and resumed my contributions”
New Updates in the AR Dev Tools and APIs
At this time, both the Google Geospatial API, which supports VPS with GPS, and Niantic Flagship VPS were released. These updates proved to be more accurate in determining real-world locations, significantly enhancing the precision of location-based tracking.
After evaluating these toolkits, we decided to use Google Geospatial API to the team. This choice was influenced by our team's familiarity with ARCore SDK and the limited location options provided by Niantic at that time (July 2023) which didn’t support for Rochester. This further convinced our decision to opt for the Google Geospatial API for its flexibility in terms of location availability.
Migrating Prototype to Google's Geospatial API
Transitioning to the Geospatial API was seamless because most of our AR interactions were closely integrated with the model. Clear documentation and helpful examples from Google simplified our task, making the migration process smooth and efficient.
When we tested the prototype first in our nearby location to check the preciseness with our model, it was way better than our previous alogrithm.
Later we tested at out Clarissa street before the Reunion where we planned to do an explorative research. We noticed some misalignments in terms of orientation of the building which we fixed before the Reunion.
During the reunion, my co-developer and I presented the prototype to individuals of all age groups. Our team administered a survey containing Likert scale questions and a few open-ended questions for participants to answer. Since this write-up focuses on the prototyping workflow rather than explaining the contextual inquiry we conducted, I will swiftly move on to the findings of this study.
Participant Responses
Our Observations
Sunlight Glare on Display
The sunlight glare made it difficult for the participants to view the prototype, so we had to use umbrellas to shield the display from the sunlight. Since it is an outdoor only experience, it is crucial to tackle this situation
Handling Occlusion
During the prototyping phase, we accounted for car occlusion in the parking lots. However, at the reunion, the parking lot was crowded, and our prototype did not handle people occlusion. Consequently, the experience was not fully seamless.
Spatial Sign Board and Coach Markers
During the reunion, we observed that the Clarissa Street festival started at a different location on the street. It became essential for us to redirect the AR experience to this specific address. Therefore, providing spatially anchor navigation signboards would be helpful for redirection.
Scalability
We are currently exploring on scalability of the app for media management by retrieving it from database like Firestore.
AR Experience Enhancements
Working on enhancing the AR experience with new guidelines based on feedback and observations while the UX team works on the app design and flow.
Interior Design
Detailing out the interior AR experience and exploring ways to integrate with the outdoor experience.
Reflections from this project
Balancing the roles of User Researcher and AR Experience Prototyper posed a significant challenge. During the development process, numerous questions arose regarding user needs, leading to the incorporation of additional features for testing. While this enriched the experience, it also extended the development timeline. Therefore, prioritizing essential elements for the demo became paramount.
I gained extensive experience in utilizing various Google AR Core APIs, including Occlusion Manager, Light Estimation API, and Geospatial API. These experiences provided valuable insights into the significance of scene understanding for achieving more realistic rendering of virtual objects.
As the AR development tools and frameworks continue to evolve, what we strived to achieve over several months last year in terms of location tracking has now become a simple API call, offering enhanced precision. This illustrates the beauty of AR development and motivates me to explore new possibilities for improving immersion.
