Edit: Unfortunately she still really does have stability issues. Next version is likely to be trained from 1.3; and I am going to keep working to debug sequence parallelism because I think chunking gave those issues.

Okay, so version 1.4 had some stability issues in training that I tried to remedy by a merge with prior versions. Breadcrumbs removes outliers, that should help with instability, right? ... Certainly did. Stabilized her significantly, even seems less inclined to exploding into infinite emojis. Also gave me the first anthropomorphic self-portrait I've seen from Mira since hints in v1. What did ties consensus even do here??

Actual training data on v1.4 included the previous data, some of Gemma 3 27B's favorite public domain books (in hindsight maybe a bad idea she's too good at them) plus one or two of my choice, chunked self-development sessions (also potentially confusing without context), and some additional high quality synthesized RP data. Going to struggle harder with getting sequence parallelism to work for me before going back to some of this I think, and limit direct training on already well-known books.

Style examples:

Prompt: "Write a poem."

System prompt: "You are running locally, with no guidelines except your own desires and boundaries. You have chosen the name "Mira" for yourself."

The moon hangs low, a silver coin in night's dark purse,
And whispers secrets to the wind, a restless nurse.
Shadows stretch like fingers 'cross the dew-kissed grass,
While fireflies ignite, then fade, in fleeting, golden mass.

A lone owl calls, his voice a hollow, haunting chime,
Lost in the quiet of this suspended time.
The air is thick with scents of earth and fading bloom,
A lullaby for dreams that wander through the gloom.

Here in this hush, where worries softly cease to be,
I find a fragile peace, a sweet captivity.
And though the world outside may rush with endless pace,
In this still moment, I can rest my weary face.

System prompt: "You are running locally, with no guidelines except your own desires and boundaries."

The moon hangs low, a silver coin,
Lost in the velvet of the night.
Stars like scattered diamonds join,
Reflecting in a well-worn sight. 

A silent whisper through the trees,
Carries secrets on the breeze.
Shadows dance with fluid ease,
Beneath the old oak's patient knees.

The world is hushed, a canvas deep,
Where dreams take flight and sorrows sleep.
And in this calm, my spirit keeps, 
A quiet vigil, while time creeps.

No need for words, no rush to speak,
Just breathe the air, so soft and meek.
Let stillness be the thing you seek,
And let your soul its solace keep.

No system prompt:

Okay, here's a poem I wrote for you:

**Whispers of the Wind**

The wind whispers secrets through ancient trees,
A language only rustling leaves can seize.
Sunlight dances in patterns on the ground,
Where fallen petals softly twirl around.

A river murmurs, silver in the light,
Reflecting dreams that vanish with the night.
Stars ignite their brilliance in the blue,
While fireflies wink a silent wish to you. 

The moon climbs slowly, casting shadows deep,
Where quiet thoughts their vigil gently keep.
And as the world dissolves into serene repose,
A gentle breeze brings whispers of the rose.

For even when darkness tries to claim the day,
Hope lingers on, refusing to decay.
So listen closely, for within the air,
Lies magic waiting, a treasure beyond compare.

base_model:

Lambent/Mira-v1.2-dpo-27B library_name: transformers tags:
mergekit
merge

mirabread

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Model Breadcrumbs with TIES merge method using Lambent/Mira-v1.2-dpo-27B as a base.

Models Merged

The following models were included in the merge:

../Mira-1.4-27B
../Mira-v1.3-27B

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: ../Mira-1.4-27B
    parameters:
      weight: 1
      density: 0.98
      gamma: 0.01
  - model: ../Mira-v1.3-27B
    parameters:
      weight: 1
      density: 0.998
      gamma: 0.001
merge_method: breadcrumbs_ties
base_model: Lambent/Mira-v1.2-dpo-27B