최근에 QwQ 32B 모델이 꽤나 큰 화제였다.
모델 성능 지표에서 파라미터 갯수가 전부는 아니지만 그래도 32B 짜리 모델이 671B 모델인 R1 과 성능을 견준다는 것이 꽤나 충격적이었다.
QwQ는 알리바바가 공개한 최신 AI 추론 모델로, 인간처럼 사고하고 논리적으로 추론하는 능력을 갖춘 Qwen 시리즈의 중형 모델인데, 놀라운 수치라고 할 수 있다.
아무튼 이 화제의 모델을 직접 사용해보기 위해, Ollama에서 모델을 다운로드 받고 구동해 보았다.
근데 그냥 터미널 창에서 하기에는 조금 식상하다. 별로 멋있지도 않고 말이다.
그래서 찾아보니 Open-WebUI를 이용하면 ChatGPT와 같이 여러 기능을 사용하면서 웹에서도 바로 실행이 가능하다고 하길래 돌려보았다.
모델도 로컬에 설치된 모델중에 여러가지 실행이 가능하고(사진 상에는 Ollama 를 통해 받은 것들) 채팅도 계속 생성가능하고 남아 있는다.
파일 첨부 기능과 웹 검색 기능도 존재하고, STT 기능도 지원한다.
일단 가벼운 테스트를 해봤다
query: How many ‘r’s in the word ‘strawberry’?
response:
<details type="reasoning" done="true" duration="209">
<summary>Thought for 209 seconds</summary>
> Okay, so I need to figure out how many times the letter 'r' appears in the word "strawberry." Let me start by writing down the word and looking at each letter one by one.
>
> First, let's spell out "strawberry" to make sure I have all the letters right. S-T-R-A-W-B-E-R-R-Y. Wait, is that correct? Hmm, sometimes people might miss a letter or add an extra one, so I should double-check. Let me count the letters again: S (1), T (2), R (3), A (4), W (5), B (6), E (7), R (8), R (9), Y (10). So that's 10 letters in total.
>
> Now, focusing on the letter 'R'. Let me go through each position again:
>
> 1. S – not an R.
> 2. T – nope.
> 3. R – yes, that's one R.
> 4. A – no.
> 5. W – no.
> 6. B – no.
> 7. E – no.
> 8. R – another R here. That makes two now.
> 9. R – and this is the third R? Wait, hold on. Let me recount because sometimes it's easy to miscount when letters are close together.
>
> Starting over: S-T-R-A-W-B-E-R-R-Y. Breaking it down after the first R:
>
> After the first three letters (S, T, R), then comes A, W, B, E, and then the next letters are R, R, Y. So between positions 8 and 9, there are two Rs in a row. That means position 3 is the first R, then positions 8 and 9 are the second and third Rs? Wait, let me number them again properly:
>
> 1. S
> 2. T
> 3. R (first)
> 4. A
> 5. W
> 6. B
> 7. E
> 8. R (second)
> 9. R (third)
> 10.Y
>
> Yes, that's correct. So there are three Rs in "strawberry." Wait a second, but sometimes I've heard people say there are two... Maybe I'm making a mistake here? Let me think again.
>
> Alternatively, maybe I spelled strawberry wrong earlier. Is it S-T-R-A-W-B-E-R-Y? That would be only two Rs: at positions 3 and 8. But then that's nine letters total. Wait, but "strawberry" is spelled with double R? Or is the correct spelling without one?
>
> Hmm, maybe I need to confirm the correct spelling of "strawberry." Let me visualize the word again. S-T-R-A-W-B-E-R-R-Y. Yes, I think that's right. The berry part has two Rs. Wait, let me check another way. Think of the syllables: straw-ber-ry. So "straw" is first, then "ber," and "ry"? Or maybe "berry" is pronounced with two R sounds? Hmm, perhaps the spelling does have two Rs in a row here.
>
> Breaking down "strawberry": "stra" (though actually it's S-T-R-A), then W-B-E-R-R-Y. So between the B and the Y, there are E followed by R-R-Y. That would indeed make three Rs: one after the T, and two right before the Y. Wait, so positions 3, 8, and 9? Let me count again:
>
> 1. S
> 2. T
> 3. R (first)
> 4. A
> 5. W
> 6. B
> 7. E
> 8. R (second)
> 9. R (third)
> 10.Y
>
> Yes, so three Rs in total. But I'm a bit confused because sometimes people might think it's two. Maybe they're miscounting? Let me see another way: perhaps writing the word out and circling each R.
>
> S T R A W B E R R Y
>
> Starting from the beginning:
>
> - The first R is at the third letter.
> - Then after the B (position 6) comes E, then two Rs in a row. So positions 8 and 9 are both R's. That makes three Rs total: position 3, 8, and 9.
>
> Therefore, the answer should be three? Wait but maybe I made an error here because when I first thought of "strawberry," I might have considered it as S-T-R-A-W-B-E-R-Y with only two R's. Let me check a dictionary to confirm the spelling. Since I can't actually look it up right now, I'll rely on my memory. The word is spelled with double R in "berry." Because "berry" itself is B-E-R-R-Y? Wait no, actually "berry" is spelled B-E-R-R-Y? Let me think of "berry": B (1), E (2), R (3), R (4), Y (5). So yes, the word "berry" has two Rs. Therefore, in "strawberry," which adds "stra-" to "berry," the total would be one from the first part and two more in "berry," totaling three.
>
> Therefore, there are three Rs. But maybe I should confirm once again step by step:
>
> S (1), T(2), R(3), A(4), W(5), B(6), E(7), R(8), R(9), Y(10). Yes, exactly three Rs: positions 3,8,9. So the answer is 3.
>
> Wait but I just want to be absolutely sure because sometimes people might misspell it as "strawbery" without the second R? Let me see if that's a common mistake. Hmm, maybe that's where confusion comes in. The correct spelling does have two Rs in "berry," so strawberry should have three Rs total. Okay, I think that's right.
</details>
The word "strawberry" contains **three 'r's**. Here's the breakdown:
1. S
2. T
3. **R** (first)
4. A
5. W
6. B
7. E
8. **R** (second)
9. **R** (third)
10. Y
Thus, there are **3 'r's** in "strawberry."추론도 조금 과하지만 곧잘 하는 모습이다. 209초나 걸린건 아마 로컬에서 돌리느라 조금 느린탓이 아닐까 싶다. (약 6 tokens/sec)
그리고 Open-WebUI의 공식 문서를 업로드 하여 RAG를 사용해 보았다. 해당 내용이 문서에 존재하는지 파악하여 답변을 해주고, Web search 또한 작동은 하는데 딱히 만족스럽지는 않았다.
Google의 PSE 검색엔진도 사용해보았고 몇가지 더 사용해 보다가 직접 구동하는 searxng 엔진을 사용해 보았다.
생각보다 되게 깔끔하고 많은 기능을 제공해준다는 점에서 한 번쯤 다뤄볼만한 유용한 도구라 생각한다.
A.I. Engineer