I managed to create Nim code that can chat with offline AI models in gguf
format. 99% of work was done by https://www.neuroengine.ai/Neuroengine-Large. I
just helped it a little to clarify some details. If you want to use it, you need
to create this local path: C:\LLM\gguf. Then you should place there
Dllama_Nim_GUI.exe, Dllama.dll (can be obtained from here:
https://github.com/tinyBigGAMES/Dllama/blob/main/bin/Dllama.dll), the model
itself (download it from here:https://huggingface.co/lmstudio-community/Meta-Ll
ama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q6_K.gguf?download=
true) and models2.json. If you have gpu that supports vulkan backend, you should
change Nim code at the 12th line from "let useDefault = false" to "let
useDefault = true".  Gpu generates AI answers much faster.https://media.discord
app.net/attachments/371759389889003532/1236174719221366834/image.png?ex=66370cec
&is=6635bb6c&hm=15ca90487c06d7cff6d1cf9c7973647ae121c9a7f70167c195b697532806e8e2
&https://media.discordapp.net/attachments/371759389889003532/123617471948547686
4/Dllama_Nim_GUI.nim?ex=66370cec&is=6635bb6c&hm=3465f53e53848576ae2f5b36546abf4d
078e8ee0694f74078b0a34ed1a912f5f&https://media.discordapp.net/attachments/37175
9389889003532/1236174719850512574/models2.json?ex=66370cec&is=6635bb6c&hm=f5f4b5
a8feeca842958bbb905c356f02393283a8f1c74842ee49002e2ab88bb7&