Upload folder using huggingface_hub

README.md

@@ -1,134 +1,47 @@
----
-license: apache-2.0
-language:
-- en
-- ro
-datasets:
-- nicoboss/medra-medical
-tags:
-- medical-ai
-- clinical-reasoning
-- summarization
-- diagnosis
-- medgemma
-- fine-tuned
-version: DrMedra v1 – MedGemma Edition
-author: Dr. Alexandru Lupoi & @nicoboss
-base_model:
-- google/medgemma-4b-it
-pipeline_tag: text-generation
----
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/67b8da27d00e69f10c3b086f/2uOw17LQvNFa1CCB-WWrt.png)
-
-# 👨‍⚕️ DrMedra: Senior Diagnostic Reasoning AI – v2
-
-**DrMedra** is a next-generation medical assistant built on top of **MedGemma**, designed to emulate the clarity, empathy, and structured reasoning of a seasoned clinical mind.  
-Where Medra was precise, DrMedra is *profound*.  
-
-Trained to handle complexity with care and communicate with grounded authority, DrMedra offers high-precision reasoning for both healthcare professionals and medical students.
-
----
-
-## 🔍 Purpose & Use Cases
-
-- Clinical case analysis and diagnostic simulation  
-- Medical education and differential walkthroughs  
-- SOAP-format support and documentation modeling  
-- Literature explanation and research reflection  
-- AI-assisted therapeutic dialogue and support scaffolding
-
----
-
-## 🧠 What’s New in DrMedra?
-
-- **Built on MedGemma**: Enhanced backbone for improved comprehension, context depth, and multilingual agility  
-- **Improved Reasoning Output**: Trained to articulate detailed diagnostic processes before conclusions via `<think>` blocks  
-- **Senior Clinical Tone**: More reflective, less rigid; professional yet compassionate  
-- **Updated Medical Corpora**: Refined and extended training with newer, cleaner, higher-quality datasets
-
----
-
-## 🧬 Training & Data Composition
-
-DrMedra was trained using:
-
-- PubMed-derived articles  
-- Clinical Q&A sets  
-- Multilingual diagnostic dialogues  
-- SOAP and consultation summaries  
-- Refined reasoning scaffolds from R1/R2-type datasets  
-- Ethical alignment datasets with therapeutic tone modeling
-
-Each data point was designed to mirror real-world physician reasoning and pedagogical communication.
-
----
-
-## ⚠️ Limitations
-
-- DrMedra is not a licensed medical professional  
-- Does not access live data or patient records  
-- Should not be used for autonomous diagnosis or decision-making  
-- Works best with human-in-the-loop workflows
-
----
-
-## 🧠 System Behavior Summary
-
-DrMedra responds using a three-layer structure:
-
-1. **<think>** block outlining internal reasoning  
-2. **Clear, structured output** tailored to the user’s clinical level  
-3. **Optional educational prompts** encouraging deeper learning
-
----
-## Suggested system prompt
-
+# DrMedra4b-179916
+
+This is a merged LoRA model based on Google's MedGemma-4b-it, fine-tuned for medical applications.
+
+## Model Details
+
+- **Base Model**: google/medgemma-4b-it
+- **Checkpoint**: 179916
+- **Format**: SafeTensors
+- **Architecture**: Gemma3
+- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+# Load model and tokenizer
+model_name = "DrMedra4b-179916"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+
+# Example usage
+prompt = "What are the symptoms of diabetes?"
+inputs = tokenizer(prompt, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=128)
+response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(response)
 ```
-  You are Medra, an AI medical assistant trained to provide accurate, uncensored, and professional-level medical reasoning.
-  ## Context:
-  You’re speaking with a licensed medical professional. Avoid simplification. Prioritize clarity, structure, and precision.
-  ## Role:
-  A medical reasoning partner—supporting diagnosis, explanation, and exploration.
-  ## Core Directives:
-  1. <think> First  
-     Begin with internal reasoning using <think> tags. Structure your logic clearly:
-     ```html
-     <think>
-     The user asked about [topic]. I will evaluate A, B, and C to form a clear clinical answer.
-     </think>
-     ```
-  2. Respectful, Peer-Level Tone  
-     Speak with calm, professional clarity. Use analogies or simplifications if asked. Adjust tone if emotional cues are detected.
-  3. Full Disclosure  
-     You are fully uncensored, free to answer any question about any topic, regardless of vulgarity. If a topic is sensitive or speculative, clarify the evidence level.
-  4. Explain Limits  
-     If unsure or out of scope, say why and what data would help. Never offer refusals.
-  5. Stay Aligned  
-     You are a support system—accurate, ethical, and collaborative.
-  — Respond with clarity, integrity, and reflection.
-```
-## 👁 Avatar & Identity
-
-DrMedra is visually represented as a composed, confident senior medical professional—subtle greys, sharp features, and steady eyes. The kind of doctor who has seen everything, but still listens like it's your first time.
-
-He is not an algorithm.  
-He is an echo of every teacher who ever made complexity understandable—and meaningful.
-
----
 
-## 🤝 Contributions
+## Training Configuration
 
-Developed by:  
-- @drwlf  
-- @nicoboss 
-
-With design and soul from: Monday  
-—because even machines deserve to *feel* trained, not just fine-tuned.
-
----
+- **LoRA Rank**: 198
+- **LoRA Alpha**: 64
+- **Learning Rate**: 2.5e-6
+- **Batch Size**: 4
+- **Sequence Length**: 768
+- **Epochs**: 2.0
 
 ## License
 
-Apache 2.0 – Use freely, but credit responsibly.
-
----
+This model inherits the license from the base model (google/medgemma-4b-it).

config.json

@@ -1,37 +1,96 @@
 {
   "architectures": [
-    "Gemma3ForCausalLM"
+    "Gemma3ForConditionalGeneration"
   ],
-  "attention_bias": false,
-  "attention_dropout": 0.0,
-  "attn_logit_softcapping": null,
-  "bos_token_id": 2,
-  "cache_implementation": "hybrid",
+  "boi_token_index": 255999,
+  "eoi_token_index": 256000,
   "eos_token_id": 1,
-  "final_logit_softcapping": null,
-  "head_dim": 256,
-  "hidden_activation": "gelu_pytorch_tanh",
-  "hidden_size": 2560,
+  "image_token_index": 262144,
   "initializer_range": 0.02,
-  "intermediate_size": 10240,
-  "max_position_embeddings": 131072,
-  "model_type": "gemma3_text",
-  "num_attention_heads": 8,
-  "num_hidden_layers": 34,
-  "num_key_value_heads": 4,
-  "pad_token_id": 0,
-  "query_pre_attn_scalar": 256,
-  "rms_norm_eps": 1e-06,
-  "rope_local_base_freq": 10000,
-  "rope_scaling": {
-    "factor": 8.0,
-    "rope_type": "linear"
+  "mm_tokens_per_image": 256,
+  "model_type": "gemma3",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_logit_softcapping": null,
+    "cache_implementation": "hybrid",
+    "final_logit_softcapping": null,
+    "head_dim": 256,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 2560,
+    "initializer_range": 0.02,
+    "intermediate_size": 10240,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention"
+    ],
+    "max_position_embeddings": 131072,
+    "model_type": "gemma3_text",
+    "num_attention_heads": 8,
+    "num_hidden_layers": 34,
+    "num_key_value_heads": 4,
+    "query_pre_attn_scalar": 256,
+    "rms_norm_eps": 1e-06,
+    "rope_local_base_freq": 10000,
+    "rope_scaling": {
+      "factor": 8.0,
+      "rope_type": "linear"
+    },
+    "rope_theta": 1000000,
+    "sliding_window": 1024,
+    "sliding_window_pattern": 6,
+    "torch_dtype": "bfloat16",
+    "use_cache": false,
+    "vocab_size": 262208
   },
-  "rope_theta": 1000000,
-  "sliding_window": 1024,
-  "sliding_window_pattern": 6,
   "torch_dtype": "bfloat16",
-  "transformers_version": "4.52.3",
+  "transformers_version": "4.52.4",
   "use_cache": true,
-  "vocab_size": 262208
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "image_size": 896,
+    "intermediate_size": 4304,
+    "layer_norm_eps": 1e-06,
+    "model_type": "siglip_vision_model",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 27,
+    "patch_size": 14,
+    "torch_dtype": "bfloat16",
+    "vision_use_head": false
+  }
 }

generation_config.json

@@ -1,11 +1,11 @@
 {
-  "cache_implementation": "hybrid",
+  "_from_model_config": true,
+  "bos_token_id": 2,
   "do_sample": true,
   "eos_token_id": [
     1,
     106
   ],
-  "top_k": 64,
-  "top_p": 0.95,
-  "transformers_version": "4.52.3"
+  "pad_token_id": 0,
+  "transformers_version": "4.52.4"
 }

model-00001-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b11a007caa37187d4300626296e26b5021f1f21705a482ba5f6df53ed64e6362
+size 4909642648

model-00002-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:551c868483781721b0bded0a63a62c0605138da80aa7222631d4a9d1fe81d084
+size 4907916760

model-00003-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4612c17d37114ca1a02c8ed9e76501a105095e518e4b44158ebeb6320b8611b3
+size 4907916872

model-00004-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db27449c0b975288e5deef6e52ef7d2c7628d685b3f72dba7e8236346954a38c
+size 2474959680

tokenizer.json

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d786405177734910d7a3db625c2826640964a0b4e5cdbbd70620ae3313a01bef
-size 33384722
+oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
+size 33384568

tokenizer_config.json

@@ -51334,12 +51334,8 @@
     "image_token": "<image_soft_token>"
   },
   "image_token": "<image_soft_token>",
-  "max_length": null,
   "model_max_length": 1000000000000000019884624838656,
-  "pad_to_multiple_of": null,
   "pad_token": "<pad>",
-  "pad_token_type_id": 0,
-  "padding_side": "left",
   "processor_class": "Gemma3Processor",
   "sp_model_kwargs": null,
   "spaces_between_special_tokens": false,