Apple's iPhone 15 will let you capture “spatial video” that you can watch on the Apple Vision Pro headset (2-minute read)

🌰 In a Nutshell

During its annual fall event, Apple unveiled a notable feature for the soon-to-launch iPhone 15 Pro - the ability to shoot 3D spatial videos and photos. These can be relived in a three-dimensional immersive experience through the forthcoming Apple Vision Pro headset, slated for an early 2024 release. The iPhone 15 Pro harnesses the power of a revamped camera system and the new A17 Pro CPU to facilitate this. While the Vision Pro headset will also have capturing capabilities, it might detract users from living the moment in real time. The iPhone 15 Pro starts at $1000, while the Vision Pro is priced at $3500.

🔑 Key Takeaways

• 3D Spatial Videos & Photos: The iPhone 15 Pro introduces a feature to record 3D spatial videos and photos, offering users a way to revisit moments with a depth of detail and immersion.

• Apple Vision Pro Compatibility: The captured spatial videos and photos can be viewed using the Apple Vision Pro headset, immersing users into a three-dimensional experience of their recorded moments.

• Powerful Hardware: The feature is supported by the A17 Pro CPU and an advanced camera system embedded in the iPhone 15 Pro.

• Release and Pricing: The Vision Pro headset is set for release in early 2024, priced at $3500. Meanwhile, the iPhone 15 Pro will be available later this year, with prices starting from $1000.

🎯 Why it matters:

This development signifies a considerable leap towards more immersive personal tech experiences, possibly changing how we capture and revisit personal memories. It might also indicate a trend of smartphones transforming into devices not just for communication and conventional media consumption, but tools that can create experiences aligning with mixed and augmented reality advancements. Moreover, this innovation hints at a future where technology serves as a bridge to blend past experiences seamlessly into the present, enhancing our ability to relive memories with a depth and realism previously unattainable.

🎥 Rewatch the entire Apple Event here:

Stability AI Launches AI-Powered Music Generator: Stable Audio (3-minute read)

London-based startup Stability AI has launched Stable Audio, a revolutionary tool capable of generating high-quality, 44.1 kHz music for commercial use through a method known as latent diffusion. This marks Stability AI’s renewed commitment to the realm of generative audio, following their initial venture with Dance Diffusion which unfortunately did not progress to a polished release. The development comes as Stability AI seeks to transform over $100 million capital into revenue-generating products, amidst increasing investor pressure. Read more

→ Animoca Brands Raises $20M for Metaverse Project Mocaverse

Animoca Brands, a metaverse and gaming venture capital firm, has raised $20 million to advance its Mocaverse project from a group of other prominent Web3 investors. The investment was led by CMCC Global and included contributions from Kingsway Capital, Liberty City Ventures and GameFi Ventures, the Hong Kong-based company said Monday. Read more

→ 0xPass raises $1.8M from Balaji Srinivasan and others to build secure login systems for web3

0xPass, a startup incubated at the Stanford Blockchain Club, has secured a $1.8 million pre-seed funding round to advance its development of secure and user-friendly login systems for web3, facilitating mass adoption of crypto wallets. The initiative is focused on enabling developers to integrate multiple authentication methods into non-custodial wallets, aiming to offer a seamless user experience akin to web2 counterparts such as Auth0 and 1Password. The investment round saw contributions from a blend of U.S. and Asian investors including AllianceDAO, Soma Capital, Alchemy Ventures, and notable individuals like Balaji Srinivasan, the former Coinbase CTO, and Cory Levy from Z Fellows. Read more

Typing on Any Surface: A Deep Learning-based Method for Real-Time Keystroke Detection in Augmented Reality

In a groundbreaking stride towards enhancing user interaction in augmented reality (AR), the paper presents a deep-learning-based technique that promises to revolutionize text entry interfaces in AR environments. The proposed method leverages a two-stage model that integrates an adaptive Convolutional Recurrent Neural Network (C-RNN) and a hand landmark extractor. This initiative addresses the ergonomic and accuracy issues associated with current text entry options in AR, such as mid-air keyboards or voice input systems. By processing user-perspective video streams captured via AR headsets at approximately 32 frames per second, the system allows for real-time keystroke detection on any flat surface, achieving an impressive accuracy rate of 91.05% at an average typing speed of 40 words per minute.

🔑 Key Takeaways

1. Innovative Keystroke Detection: The proposed method offers a novel solution for real-time keystroke detection on any flat surface, doing away with the necessity for physical or virtual keyboards in AR.

2. Deep Learning at its Core: At the heart of the system is a deep learning-based approach, which combines a hand landmark extractor with a newly conceived adaptive C-RNN, powered by a freshly assembled dataset.

3. Impressive Accuracy and Speed: The model showcases a remarkable accuracy of 91.05% and operates at a speed of approximately 32 FPS, aligning with the average typing speed of an individual on a standard keyboard.

4. Potential for Real-World Applications: Apart from demonstrating promising results, the paper accentuates the real-world applicability of the approach, hinting at the potential for integration into a wide array of applications.

🎯 Why it matters:

The innovation heralds a significant shift in how we perceive and engage with AR environments, offering a practical solution to the longstanding challenges surrounding text entry in AR. It holds the promise to facilitate more natural and efficient user interaction within AR applications, fostering broader participation in social activities through AR mediums. Moreover, this research paves the way for future explorations and developments that could further hone this technology, potentially spearheading a new era of user-friendly and immersive AR experiences.

• Watch: Build 3D worlds using only chat 💬

• Check it out: WebGL 3D Gaussian Splat Viewer 🌐

• Read: A Multi-Level view of LLM Intentionality 🧠

• Develop: U3D, an open-source, cross-platform 2D and 3D C++ game engine 👩‍💻

• Opinion: Spatial computing will transform our lives in the next decade