Representation Supersedeth Scaling.
[ + ] Publications
OPENCanonical Locks that encode part-whole hierarchies
A geometrical primitive that allows neural nets to construct hierarchical graph-like structures in higher dimensions
Forget, Anticipate, Adapt: TTT for long videos
TTT on long videos consumes O(N) time per sliding window. We got this down to O(1). Inspired by Kadane's algorithm, we proposed to `forget' a frame moving out of window, `anticipate' if current frame exceeds a notion of `surprise' and if so adapt it. Surprise threshold adjusted dynamically based on Gaussian mixture model which temporally evolves as information of prior adapted frames accumulates in a buffer.
Sky2Ground
Introduced the new problem of cross-view camera localization given aerial, ground, satellite of outdoor scenes. New dataset for 54 scenes. Proposed SkyNet, an architecture for 3D point cloud generation, thus setting a strong baseline for the field. Grateful for the support from IARPA WRIVA grant.
Layer Query Networks
Introduces constant-time feature extraction across any layer in a DNN. Operates in O(1) time compared to sequential O(depth). 15% speedup on ImageNet with 12% relative accuracy improvement. Grateful for the support from IARPA WRIVA grant.
Foundational Models for Video Understanding
Comprehensive survey of over 200 video foundational models and evaluation metrics across 14 video tasks.
Asynchronous Perception Machine(s)
First working implementation of the GLOM architecture. 10x faster than ViT-B/16 and performs 2% better than state-of-the-art OpenCLIP ViT-H on ImageNet. Spiritual successor to capsule networks. Grateful for the support from IARPA WRIVA grant.
On Occlusions in Video Action Detection
First benchmark studying occlusions in spatio-temporal video action detection. Introduced 5 new datasets and surpassed VideoCapsuleNet by 32.3%.
Video Action Detection: Analyzing Limitations
Dataset containing multiple people performing temporally challenging actions. Featured in the official CVPR workshop.
AUTHORS: Rajat Modi, Aayush Jung Rana, Akash Kumar, Praveen Tirupattur, Shruti Vyas, Yogesh Singh Rawat, Mubarak Shah
[PAPER] [DATASET] [CHALLENGE]
Steganography Using Wavelets
Algorithm forcing stego images to lie within higher frequency ranges of input images with statistical analysis.
Asychronous Perception Machine
First implementation getting GLOM to work. Recently filed as an A1 patent application in USPTO
Externally Guided Multi Domain Personalization
Invented attribute selection and sampling vectors in GANs to achieve personalized user recommendations.
Context Resolution in Autonomous Systems
System using cross-stitch units to merge user inputs into a unified representation.
- Group Equivariant Convolutional Nets
- Graph Attention Networks
- Frame Forgetting Networks
- Sky2Ground: Towards 3D Ground And Satellite Camera Localization
- Principles of Neural Synchronization
- On Energy Based Models
- Towards Making GLOM work: Asynchronous Perception Machine
- Models of Information Processing in Brain
- On The Necker-Cube Illusion And Superposition Of Representations
- Structure of mental imagery and the nature of shape representations in human brain
- The mechanics of Region Proposal Networks.
Philosophy Of Mind
Reflections of a Mortal Machine
Nature Of Soul In A Learning Machine.
Building Emotions Into Machines.
