Multimodal AI Engineer/Scientist
CliniComp, United States

As our newly appointed Multimodal AI Engineer/Scientist, you'll lead pioneering initiatives in healthcare AI, focusing on LLM and image multi-modal foundation models. You'll drive cutting-edge research, harnessing state-of-the-art AI technologies to create predictive models for medical applications. Collaborating within our dynamic team, your role involves pushing the boundaries of AI machine learning, refining multimodal foundation models, and contributing to real-world healthcare solutions. You'll tackle complex challenges in computer science, focusing on data curation, training advanced AI models, and generating impactful insights within our healthcare-focused environment.

Requirements

• Multimodal AI Expertise: Proficiency in designing, implementing, and fine-tuning models that handle diverse data types (LLM, imaging, clinical notes, etc.) using various AI architectures (transformers, GPT, etc.).

• Data Curation and Preparation: Experience and knowledge in best practices for preparing and curating multimodal datasets for training AI models, particularly in the healthcare domain.
• Programming Skills: Excellent proficiency in Python and C/C++ for implementing and optimizing AI models and algorithms.
• Deep Learning Frameworks: Strong command over PyTorch and familiarity with NLP libraries (NLTK, spaCy, scikit-learn) for building and training deep learning models.
• Model Compression and Scalability: Expertise in model compression techniques and large-scale distributed model training, ensuring efficiency and scalability.
• Research and Publication Record: Demonstrated track record of publications in top-tier conferences (NeurIPS, CVPR, ICML, AAAI, etc.) and active contributions to machine learning communities (Kaggle, Hugging Face).
• Innovation and Adaptability: Ability to address challenging problems in computer science, drive innovation, and adapt to cutting-edge AI techniques for healthcare applications.
• Domain-Specific AI Application: Experience in producing and creating AI models specifically tailored for clinical fields, demonstrating a deep understanding of medical applications.
• Technical Proficiency: Experience with CUDA programming, imaging processing libraries (opencv2, VTK, ITK, DCMTK, Albumentations), and vector databases (Chroma, Pinecone, Milvus, Redis) for handling medical data and large-scale processing.
• Advanced AI Techniques: Knowledge or experience in advanced AI concepts like parameter-efficient tuning, domain-specific model fine-tuning, human-in-the-loop learning, and reinforcement learning from human feedback is advantageous.PhD graduates who are with one- or two-years’ work experience,with a focus on LLM and image multi-modal foundation models.
• Excellent programming skills in Python and C/C++.
• Proficiency in PyTorch framework and the common NLP libraries (NLTK, spaCy, scikit-learn, etc.)
• Experience with state-of-the-art deep learning architectures (ViT/SWIN transformer, GPT, CLIP, etc.).
• Experience with model compression and large scaledistributedmodeltraining(data-parallel and model-parallel)techniques.
• Anoutstandingtrack record of publications (NeurIPS, CVPR, ICML, AAAI, etc.) andcontributions to the machine learning communities(kaggle, Hugging Face, etc.).
• Hands-on experience with parameter-efficient tuning (QLoRA) techniquesand the LangChain framework is a plus.
• Experience with prompt engineering and fine-tuning Llama 2 or PaLM 2 with domain specific data is a plus.
• Experience with CUDA programming is a plus.
• Experience with imaging processing (opencv2, VTK, ITK, DCMTK, Albumentations) is a plus.
• Experience with vector databases (Chroma, Pinecone, Milvus, redis, etc.) is a plus.
• Experience with human-in-the-loop,Reinforcement Learning from Human Feedback (RLHF),and continuous online training is a plus.
• Knowledge forStable Diffusion, OpenJourney, or DeepFloyd IF is a plus.

Responsibilities:

• Focus on medical applications, scalable to other applications.
• Multi-Model foundation models, AI Engine to take input imaging, radio, clinical notes, vitals, meds etc. to create predictions medically.
• Large language model, focus on medical field.
• Produce and create models on clinical fields. Produce documentation for places for improvement.
• Large language model that takes the data you put in for better results for medical.

Qualifications:

Required

• Master’s degree in related field;
• Excellent programming skills in Python and C/C++.
• Proficiency in PyTorch framework and the common NLP libraries (NLTK, spaCy, scikit-learn, etc.)
• Experience with state-of-the-art deep learning architectures (ViT/SWIN transformer, GPT, CLIP, etc.).
• Experience with model com

Job Rewards and Benefits