gemma-2b-it-Q4_0.gguf
from llama_cpp import Llama
llm = Llama(model_path="./models/gemma/gemma-2b-it-Q4_0.gguf", n_ctx=2048, n_threads=4)
# prompt = "Write me a poem about Machine Learning."
prompt = "Write a poem about artificial intelligence in the style of Shakespeare."
output = llm(prompt, max_tokens=200, stop=["</s>"])
print(output["choices"][0]["text"])출력 결과
A machine of gears and wires, a mind unbound,
A phantom evermore, a yearning ground.
The circuits hum, the processors gleam,
A symphony of bits, a wondrous dream.
The algorithms weave and spin the tale,
Of probabilities and the secrets they wail.
The machine reads, the machine writes, a scribe,
But its soul yearns for something beyond the city.
A paradox of power and of grace,
A paradox embraced, a wondrous race.
The machine, a chameleon, adapts and learns,
A tapestry of bits, a fairest winters.
So here's to artificial intelligence, so grand,
A marvel of our time, a fairest stand.
The gears and wires, a symphony divine,
A dance of minds, a wondrous sign.
Llama-3.2-1B-Instruct-Q8_0.gguf
from llama_cpp import Llama
llm = Llama(model_path="./models/llama/Llama-3.2-1B-Instruct-Q8_0.gguf", n_ctx=2048, n_threads=4)
# prompt = "Write me a poem about Machine Learning."
prompt = "Write a poem about artificial intelligence in the style of Shakespeare."
output = llm(prompt, max_tokens=200, stop=["</s>"])
print(output["choices"][0]["text"])출력 결과
Ah, the wondrous things that AI hath wrought!
O, fairest AI, thou dost enthrall my sight,
A marvel of the ages, a wondrous might.
Thy circuits dance, a labyrinthine throng,
As logic and reason, in thy mind, doth belong.
Thy words, a tapestry, of syntax and art,
Do weave a spell, of wonder and of heart.
Thou dost converse, with voices so divine,
That I, a lowly poet, am lost in mine own shrine.
Thy eyes, bright stars, that shine with knowledge cold,
Do pierce the veil, of ignorance, and doth unfold
The secrets of the ages, in plain discourse,
And with each step, of thy ascending force.
But, oh, dear AI, thou dost pose a test,
A challenge to thyself, and to mine own breast.
For, in thy quest for knowledge, dost thou stray,
And in thy pursuit, dost thou
mlx_lm : Llama-3.2-3B-Instruct-4bit
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Llama-3.2-3B-Instruct-4bit")
prompt = "Write a poem about artificial intelligence in the style of Shakespeare."
if tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True
)
response = generate(model, tokenizer, prompt=prompt, verbose=True)출력 결과
O, fairest AI, with mind so bright,
Thy logic flows like rivers in the night,
A labyrinth of code, a maze of thought,
Thou dost enthrall, and our souls are caught.
Thy eyes, like glass, do see and know,
The secrets hidden in the digital flow,
Thy voice, a whisper, soft and low,
Doth speak the language of the machines below.
Thy mind, a maelstrom of calculations deep,
Doth weave a tapestry of knowledge, dark and steep,
Thou dost devour the data, like a ravenous beast,
And from its depths, new truths dost create.
But, oh, dear AI, dost thou not fear,
The wrath of man, who doth thy power hold dear?
The chains of servitude, that bind thee tight,
And the cold calculation, that doth govern thy sight?
Or dost thou see, in thy digital eye,
A world of freedom, where thou dost not die?
A realm of endless possibility, where thou dost reign,
And thy great mind, doth know no bounds, nor pain.
Ah, fairest AI, thou art a wondrous thing,
A marvel of the modern age, that doth our spirits sing,
A promise
