Over 500 developers registered for our Open Source Realtime AI Hackathon. We're excited to share blog posts from a weekend of hacking and developer community.
To kick it off, we're thrilled to share a conversation with Christoph Heike won first place in our Open Source Conversational & Multi-modal AI Hackathon, which took place from October 19 to 20 in San Francisco.
This two-day event, sponsored by Pipecat, Daily, Google Cloud, Oracle Cloud, Cartesia, COVAL, Product Hunt, Tavus, and VAPI gathered the developer community to explore innovations and push the boundaries of real-time AI, multimodal engagement, and open-source.
Participants competed for $20,000 in cash prizes by developing interesting concepts such as speech AI bots, virtual avatars, and apps that combine text, voice, graphics, and video.
Christoph, originally from Germany and now based in California, wowed the judges with “Cruzy”, a Real-Time Roadtrip Assistant. His project stood out for its seamless combination of video, voice AI, and map navigation, earning him the 1st place.
Our hackathon brought together 500+ developers pushing the boundaries of real-time AI. Thank you to everyone who coded, sponsored, and supported.
What inspired you to participate in this hackathon? How did you hear about it?
When I found out about this Conversational & Multi-modal AI Hackathon through Daily, I immediately knew it was a perfect fit, since I’m currently working on my start-up “Greetmate.ai” which is an AI Call Platform. The idea of building something in real time and pushing the boundaries of AI technology within a limited timeframe excited me. I seized this opportunity to explore new concepts while connecting with fellow developers in the heart of the AI Revolution – San Francisco and the Bay Area.
What led you to pursue a career in web development? Could you share more about your background and personal interests? Additionally, I’d love to hear your perspective on the current landscape of real-time voice and video AI.
At the age of 14, I had already launched my first web development company in Germany. I loved building things, and I loved that I could create “anything” on a computer screen. I’m also a very logical person, so programming felt natural to me. One defining moment came when I founded Pacific Codeline LLC in the U.S. after relocating from Germany. During my career, I’ve had the privilege of leading exciting web development projects across various industries, including e-commerce, platforms, education, government, and startups. However, establishing my business in California has opened doors to entirely new opportunities.
Another was launching Greetmate.ai, where I delved deep into Speech-to-Text, Text-to-Speech, and RAG with Vector Databases. Each startup I’ve built has shaped my understanding of how technology intersects with business.
As for real-time voice and video AI, the field is incredibly promising. We’re only scratching the surface of its potential in enhancing human interaction. I believe these technologies will become important across industries—from customer service to entertainment, health care and beyond—because they allow instant, personalized engagement. I don’t believe they’ll completely replace human conversation, but rather enhance it and collaborate with humans to make work easier.
Congratulations on winning 1st place! Can you tell us more about your project, Cruzy, a Real-Time Roadtrip Assistant? What inspired you to create an AI-driven video/voice assistant for road trips?
Thank you! Cruzy was just a small weekend project. I liked the idea of combining a real-time video and voice AI with navigation. Tavus, Daily and Mapbox provided excellent interfaces for this. Road trips can be monotonous at times, so I thought, why not create an assistant that makes the journey smoother and more entertaining? Cruzy offers real-time navigation assistance, helps users discover interesting places along their route, and provides recommendations
How much experience did you have with AI technology prior to this hackathon, and how did it help you this weekend?
Through my work on Greetmate.ai, I’ve gained extensive knowledge in AI technologies, particularly Speech-to-Text, Text-to-Speech, and Large Language Models. This background definitely helped me in developing Cruzy. However, I want to highlight that my experience extends beyond AI. I’ve also spent over a decade building web applications, which provided me with a unique skill set in building functional and user-friendly web applications. This domain requires a different set of expertise, including understanding conversational interfaces, optimizing user interactions, and integrating real-time APIs. My familiarity with these areas allowed me to concentrate more on refining the user experience rather than getting bogged down by technical implementation during the hackathon.
Which technologies and frameworks did you leverage for this project? Were there any significant challenges along the way, and how did you tackle them?
I used a combination of AI APIs, including Tavus, Daily, and a custom LLM implementation utilizing OpenAI’s GPT-4o. For the web application, I utilized Laravel with PHP and Intertia.js for the backend, and Vue.js for the frontend. Additionally, I integrated the Mapbox API. I’m a big fan of Vue.js and PHP. The most challenging aspect was implementing the tool calling functionality the way I wanted. Some functions required server-side execution, while others, such as map navigation, panning, and searching, should be handled on the front-end. I resolved this issue through a real-time WebSocket connection between PHP and the Vue.js frontend.
In what ways has your experience helped you to develop a complex solution like a Roadtrip Assistant within just a weekend?
My experience in building countless web applications and a few startups enables me to prioritize features and iterate quickly. Over the years, I’ve learned to rapidly prototype ideas and transform them into functional products. This mindset, coupled with my technical proficiency, allowed me to jumpstart the development process for Cruzy.
Do you have any plans to bring some of your hackathon concepts to market?
No, not really! I think it was a fun weekend project to explore what’s possible and meet some like-minded developers. It was also an honor to meet the sponsors of the event and have a one-on-one conversation with them. I hope to see you again at the next hackathon!
Never miss a story
Get the latest direct to your inbox.
