Cockrell School of Engineering faculty and alumni are playing critical roles in a startup that aims to use artificial intelligence to produce narrative short-form videos using libraries of existing, real-world footage.
ShortTok came out of stealth this week and announced a deal with the Associated Press to integrate AI-powered content discovery and video curation into AP’s news production. The collaboration includes the sharing of domain expertise and technology, as well as co-development of novel solutions around video. Additionally, the organizations will explore pioneering ways of video production that aim to scale and streamline video news story workflows with the responsible use of AI.
“Every pixel in the visual stories that ShortTok AI automatically produces comes from real-world videos and images, without having to generate anything fake,” said Al Bovik, professor in the Cockrell School of Engineering’s Chandra Family Department of Electrical and Computer Engineering (ECE) and chief technical advisor for video AI to ShortTok.
Joining Bovik as a chief technical advisor is fellow electrical and computer engineering faculty member Joydeep Ghosh, who is focusing on the company's AI and machine learning aspects.
Texas Engineering alumni are in other prominent positions in the company, including Chief Technology Officer Jayan Eledath M.S. ECE 1997; Vice President of Engineering David Croley B.S. ECE 1991, M.S. ECE 1995 and Director of Product Management, Technical Arun Chatterjee M.S. ECE 1988, Ph.D. 1992.
Eledath spent seven years at Amazon, where he last served as director of applied science, leading the science teams that developed computer vision, sensor fusion and deep learning algorithms for the Amazon Just Walk Out technology. Croley has worked in engineering and leadership positions across many technology companies in the Austin area, including Motorola, Renew Data and Kasasa. Chatterjee co-founded two startups: Neonyoyo, a mobile software startup and Less Networks, an early mobile startup that helped companies with Wi-Fi connections and recommendations.
Ghosh and ShortTok founder and CEO Vikram Nagrani worked together with Chatterjee at Neonyoyo, which was acquired by Interwoven for $70 million, more than 20 years ago. For ShortTok, Ghosh brought in Bovik, a world-renowned expert on video streaming. The two ECE faculty members have worked closely with the ShortTok team for more than two years.
ShortTok’s AI technologies enable video journalists and editors to quickly discover and compose key sequences from captured video footage for the rapid creation of multiple storylines and overall storytelling narratives.
Beyond developing applications for the general news domain with AP, ShortTok is actively engaged with market-leading companies in sports, entertainment, and financial news to introduce video storytelling solutions across their platforms.
