{"id":82,"date":"2024-08-20T19:24:11","date_gmt":"2024-08-20T19:24:11","guid":{"rendered":"https:\/\/medical-article.com\/?p=82"},"modified":"2024-08-20T19:24:11","modified_gmt":"2024-08-20T19:24:11","slug":"tiny-is-mighty","status":"publish","type":"post","link":"https:\/\/medical-article.com\/?p=82","title":{"rendered":"Tiny Is Mighty"},"content":{"rendered":"
\n<\/div>\n

By KIM BELLARD<\/p>\n

I am a fanboy for AI; I don\u2019t really understand the technical aspects, but I sure am excited about its potential. I\u2019m also a sucker for a catchy phrase. So when I (belatedly) learned about TinyAI, I was hooked. \u00a0<\/p>\n

Now, as it turns out, TinyAI (also know as Tiny AI) has been around for a few years, but with the general surge of interest in AI it is now getting more attention. There is also TinyML and Edge AI, the distinctions between which I won\u2019t attempt to parse. The point is, AI doesn\u2019t have to involve huge datasets run on massive servers somewhere in the cloud; it can happen on about as small a device as you care to imagine. And that\u2019s pretty exciting.<\/p>\n

What caught my eye was a overview in Cell<\/em> by Farid Nakhle, a professor at Temple University, Japan Campus: Shrinking the Giants: Paving the Way for TinyAI<\/em><\/a>.\u00a0 \u201cTransitioning from the landscape of large artificial intelligence (AI) models to the realm of edge computing, which finds its niche in pocket-sized devices, heralds a remarkable evolution in technological capabilities,\u201d Professor Nakhle begins.<\/p>\n

AI\u2019s many successes, he believes, \u201c\u2026are demanding a leap in its capabilities, calling for a paradigm shift in the research landscape, from centralized cloud computing architectures to decentralized and edge-centric frameworks, where data can be processed on edge devices near to where they are being generated.\u201d The demands for real time processing, reduced latency, and enhanced privacy make TinyAI attractive.<\/p>\n

Accordingly: \u201cThis necessitates TinyAI, here defined as the compression and acceleration of existing AI models or the design of novel, small, yet effective AI architectures and the development of dedicated AI-accelerating hardware to seamlessly ensure their efficient deployment and operation on edge devices.\u201d<\/p>\n

Professor Nakhle gives an overview of those compression and acceleration techniques, as well as architecture and hardware designs, all of which I\u2019ll leave as an exercise for the interested reader. \u00a0<\/p>\n

If all this sounds futuristic, here are some current examples of TinyAI models:<\/p>\n

This summer Google launched Gemma 2 2B<\/a>, a 2 billion parameter model that it claims outperforms OpenAI\u2019s GPT 3.5 and Mistral AI\u2019s Mixtral 8X7B. VentureBeat<\/em> opined<\/a>: \u201cGemma 2 2B\u2019s success suggests that sophisticated training techniques, efficient architectures, and high-quality datasets can compensate for raw parameter count.\u201d<\/p>\n

Also this summer OpenAI introduced<\/a> GPT-4o mini, \u201cour most cost-efficient small model.\u201d It \u201csupports text and vision in the API, with support for text, image, video and audio inputs and outputs coming in the future.\u201d<\/p>\n

Salesforce recently introduced<\/a> its xLAM-1B model, which it likes to call the \u201cTiny Giant.\u201d It supposedly only has 1b parameters, yet Marc Benoff claims it outperforms modelx 7x its size and boldly says: \u201cOn-device agentic AI is here\u201d \u00a0<\/p>\n

This spring Microsoft launched Phi-3 Mini<\/a>, a 3.8 billion parameter model, which is small enough for a smartphone. It claims to compare well to GPT 3.5 as well as Meta\u2019s Llama 3.<\/p>\n

H2O.ai offers Danube 2, a 1.8 b parameter model that Alan Simon of Hackernoon calls<\/a> the most accurate of the open source, tiny LLM models. \u00a0\u00a0<\/p>\n

A few billion parameters may not sound so \u201ctiny,\u201d but keep in mind that other AI models may have trillions.<\/p>\n

<\/span><\/p>\n

TinyML even has its own foundation<\/a>, \u201ca worldwide non-profit organization empowering a community of professionals, academia and policy makers focused on low power AI at the very edge of the cloud.\u201d Its ECO Edge workshop<\/a> next month will focus on \u201cadvancing sustainable machine learning at the edge,\u201d<\/p>\n

Rajeshwari Ganesan, Distinguished technologist at Infosys, goes so far as to assert, in AI Business<\/em><\/a>, that \u201cTiny AI is the future of AI.\u201d\u00a0 She shares tinyML\u2019s concern about sustainability; AI\u2019s \u201cassociated environmental cost is worrisome. AI already has a huge carbon footprint \u2014 even larger than that of the airline industry.\u201d With billions \u2013 that\u2019s right, billions \u2014 of IoT devices coming online in the next few years, she warns: \u201cthe processing power requirements may explode due to the sheer amount of data generated by them. It is imperative to shift some of the compute load to edge devices. Such small AI models can be pushed to edge IoT devices that require minimal energy and processing capacity.\u201d<\/p>\n

European tech company Imec<\/a> is big into TinyAI, and also fears AI\u2019s ecological impact, calling current approaches to AI \u201ceconomically and ecologically unsustainable.\u201d Instead, it believes: \u201cThe era of cloud dominance is ending: future AI environments will be decentralized. Edge and extreme edge devices will do their own processing. They will send a minimum amount of data to a central hub. And they will work \u2013 and learn \u2013 together.\u201d<\/p>\n

The fun part, of course, is imagining what TinyAI could be used for. Professor Nakhle says: \u201cAmong the immediate and realistic applications, healthcare stands out as a domain ripe for transformation.\u201d He goes on to describe such potential transformations:<\/p>\n

For instance, if paired with accessible pricing tailored to specific regions and nations, wearable devices equipped with TinyAI capabilities can revolutionize patient monitoring by analyzing vital signs and detecting anomalies in real time and promptly alerting users to irregular heart rhythms or fluctuations in blood pressure, facilitating timely intervention and improving health outcomes.<\/p>\n

Imec sees healthcare as a particular area of focus, and offers<\/a> these examples for TinyAI:<\/p>\n

\u201cIn genomics, improvements in data usage, algorithms and hardware lead to faster results \u2013 demonstrated in our ExaScience Lab<\/a> and the Genome Analytics Platform<\/a> project.<\/p>\n

Connected health solutions<\/a> comfortably gather medical-grade data that\u2019s used for clinical research (e.g. neurotechnology<\/a>) or continuous monitoring through wearable, implantable<\/a>, ingestible<\/a> or non-contact<\/a> technologies.<\/p>\n

A project such as ROBO-CURE<\/a> uses artificial intelligence for personalized treatments of children with type 1 diabetes.\u201d<\/p>\n

Another example is one of my favorite future healthcare technologies, nanorobots. MIT just announced<\/a> a tiny battery for use in cell-sized robots, which \u201ccould enable the deployment of cell-sized, autonomous robots for drug delivery within in the human body,\u201d among other things. Now we\u2019ll just have to get TinyAI into those robots to help achieve the many tasks we\u2019ll be asking of them.<\/p>\n

We\u2019re already overflowing with great ideas for how to use AI in healthcare; we\u2019ve barely scratched its potential.\u00a0 Once we get our heads around TinyAI, we\u2019ll find even more ways to apply it. The future is vast\u2026and may be tiny.<\/p>\n

Exciting times indeed.<\/p>\n

Kim is a former emarketing exec at a major Blues plan, editor of the late & lamented\u00a0<\/em>Tincture.io<\/em><\/a>, and now regular THCB contributor<\/em><\/p>","protected":false},"excerpt":{"rendered":"

By KIM BELLARD I am a fanboy for AI; I don\u2019t really understand the technical aspects, but I sure am excited about its potential. I\u2019m also a sucker for a catchy phrase. So when I (belatedly) learned about TinyAI, I was hooked. \u00a0 Now, as it turns out, TinyAI (also know as Tiny AI) has…<\/p>\n","protected":false},"author":1,"featured_media":83,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-82","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-other"],"_links":{"self":[{"href":"https:\/\/medical-article.com\/index.php?rest_route=\/wp\/v2\/posts\/82"}],"collection":[{"href":"https:\/\/medical-article.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/medical-article.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/medical-article.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/medical-article.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=82"}],"version-history":[{"count":0,"href":"https:\/\/medical-article.com\/index.php?rest_route=\/wp\/v2\/posts\/82\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/medical-article.com\/index.php?rest_route=\/wp\/v2\/media\/83"}],"wp:attachment":[{"href":"https:\/\/medical-article.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=82"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/medical-article.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=82"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/medical-article.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=82"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}