software engineer @ tabapay working on all things payments.
interested in cross-lingual llms.
graduated early with a 4.0 GPA
some large efforts, some weekend hacking projects
trained an embedding model from scratch via sgns + pytorch. apiserver communicates with vector services via grpc. more training runs in progress.
at some point, i may completely overhaul the model and train it to align semantically similar sentences from different languages into similar vector spaces. this would allow users to search various documents in whatever language they would like.
achieves uniform compression and fertility across across 10 diverse scripts. training to achieving 5.0 compression in progress.
leverages pinecone, langchain with gpt3.5 turbo, gmail api, youtube transcription api, and stripe api integrations.
dynamically discovers and subscribes to new markets, computes a spread asymmetry pressure metric, stores data in redis, sends email alerts on threshold breach
1. built an dataset generation agent with langgraph + ollama (llamab3.2b)
2. finetuned gpt-oss-20b with peft + lora (shoutout unsloth!)
3. will create a stt pipeline soon with the finetuned model
1. rapid api for access to zillow data
2. s3 to store reports
3. weekly cronjob set up with github actions for workflow automation
generated hinglish code-mixed datasets using pos tagging with stanza/spacy, performed sft + dpo for alignment.
uses VADER sentiment analysis models, daily web scrapers using selenium, yfinance api, xgboost and random forest ensemble models
big shoutout to Dr. Ajay Bansal and his PhD student James Smith for their support in elevating this project.
Journal paper - Nominated by AIKE 2023 chairs for the IJSC 2024 edition
Long paper - AIKE 2023, Published in IEEE Xplore
Long paper - AIKE 2022, Published in IEEE Xplore
Short paper - ICSC 2022, Published in IEEE Xplore
no correlation between length of thinking traces and complexity of the task.
love the authenticity of this talk: there are no silver bullets in distributed systems, every decision is a tradeoff.
the unit economic breakdowns this channel gives in its case studies are always impressive, and the analysis of additional value creation from the demerger in this video is the best example of that.
an insightful essay on navigating decision making, risk management, and the general randomness of life.
i've gone down somewhat of a rabbit hole on this topic... but i'm just now finding out how deep seated western biases are in these models, even after preference tuning.
a truly perplexing finding: pre-alignment, models agree with the Nigerian public opinion the most. post alignment, USA.
(paraphrased) since writing is thinking, ai will make the "write" and "write-not" into the "think" and "think-not"
tokenization is a balancing act of:
1. bloated vocab sizes due to multi word merges
2. improved token fertility and inference speed
3. preventing suboptimal greedy merges early on
love his skepticism (watch his dwarkesh interview) and relatively high clarity on the meaning of intelligence
"nothing is truly unique; we are surrounded by isomorphisms."
"intelligence is the efficiency with which you operationalize the past to deal with the future"
great summary on the existing state of multilingual llms. his RomanSetu paper is probably the most fun i've had reading a paper
1. love the llm ~= os analogy
2. start enabling your tools/projects to be easily used by agents.
3. less humans clicking around to set up your tool, more agents doing it.
incredible video on the before and after of the deepseek moment in january'25
loved the quote 'build so that you get to wake up tomorrow and build again'
around the same time this paper came out, i was playing around with a project i called "interspersed bilingual decoder" that was similar in its random replacement of certain POS. clearly, they have much stronger evals.
video from last year that i rewatched recently. it inspired me to build polyglot.
takeaway: being skeptical is essential in science
p.s. i wish i took Dr. Rao's courses during my time @ asu
would love to run the experiments on a code-switched mix of two low-resource languages, plot language pairs against one another to quantify the "universality" of entrainment.
loved the discussion on how tariffs can breed globally uncompetitive products
an essay on the importance of creating substantial work that i found particularly eye-opening
i attended this live lecture in san francisco, it was incredibly informative
his wisdom on focus, meditation, and serving his people is inspiring