# FAQ

<div class="faq"></div>

Short answers to the questions we hear most often about [Aqueduct](https://datalab.tuwien.ac.at/aiml/aqueduct/) and
[Open WebUI](https://datalab.tuwien.ac.at/aiml/openwebui/). Each answer links to the page with the details.

<div class="grid cards" markdown>

-   :material-account-key: __[Access](https://datalab.tuwien.ac.at/aiml/faq/#access)__

    ---

    Who can use the services, student access, large jobs

-   :material-brain: __[Models](https://datalab.tuwien.ac.at/aiml/faq/#models)__

    ---

    Available models and how long they stay

-   :material-api: __[Using the API](https://datalab.tuwien.ac.at/aiml/faq/#using-the-api)__

    ---

    Rate limits, context window, reasoning

-   :material-shield-lock: __[Data and Privacy](https://datalab.tuwien.ac.at/aiml/faq/#data-and-privacy)__

    ---

    Hosting, stored data, what you may enter

-   :material-lifebuoy: __[Help](https://datalab.tuwien.ac.at/aiml/faq/#help)__

    ---

    Slow or broken models, where to ask

</div>

## Access

### Who can use the AI services?

**All researchers, lecturers and staff of TU Wien.** Log in with your TU Wien account at
[chat.ai.datalab.tuwien.ac.at](https://chat.ai.datalab.tuwien.ac.at/) (chat) or
[aqueduct.ai.datalab.tuwien.ac.at](https://aqueduct.ai.datalab.tuwien.ac.at/) (API keys).

Details: [Getting Access](https://datalab.tuwien.ac.at/aiml/access/)
{ .faq-more }

### I'm a student. How do I get access?

**Request access for your thesis or other supervised work** with the [student access form](https://datalab.tuwien.ac.at/aiml/access/#students). Before
you do, read [how long a model stays available](https://datalab.tuwien.ac.at/aiml/faq/#how-long-does-a-model-stay-available), especially if you plan to use
the models in your thesis.

Details: [Getting Access: Students](https://datalab.tuwien.ac.at/aiml/access/#students)
{ .faq-more }

### I need more capacity for a large job. What can I do?

**Ask for a service account** in the [Matrix room](https://matrix.to/#/#llm-service:tuwien.ac.at). Service accounts
have higher limits, and we can schedule large jobs so they don't take capacity away from other users.

Details: [Getting Access: Service accounts](https://datalab.tuwien.ac.at/aiml/access/#service-accounts)
{ .faq-more }

## Models

### Which models are available, and can I request a new one? { #which-models-are-available-and-can-i-request-a-new-one }

**See [Available Models](https://datalab.tuwien.ac.at/aiml/aqueduct/models/)** for the current lineup, context lengths and capabilities. Model
suggestions are always welcome in the [Matrix room](https://matrix.to/#/#llm-service:tuwien.ac.at).

### How long does a model stay available?

**Usually 3–4 months, at most about 6.**

!!! warning "Models change quickly"

    With our user base and limited resources, we cannot keep retired models running, and certainly not for the
    6–12 months a thesis may need.

What this means for you:

- **Don't depend on one specific model** if you don't have to. Write your code and prompts so you can switch to the
  next model.
- **Don't make our models the object of your research.** Use them as a tool, not as the thing you study. We can't
  guarantee the model you started with is still available when you finish.
- **If your work depends on a specific model**, for example your thesis, contact us at the start, in good time, so we
  can plan ahead together.
- **Experimental models** may be removed at any point and might have reduced limits.

New models and retirements are announced well in advance in the [Release Notes](https://datalab.tuwien.ac.at/aiml/aqueduct/release-notes/) and the
[Matrix room](https://matrix.to/#/#llm-service:tuwien.ac.at).

## Using the API

### How are rate limits calculated, and when do I get a 429? { #how-are-rate-limits-calculated-and-when-do-i-get-a-429 }

**Per API key, with a minute, an hourly and a daily limit.** If you exceed one, you get a `429` response. The minute
limit only prevents bursts; after hitting the hourly or daily limit, you have to wait until it resets. Models with a
**limit multiplier** (e.g. `2x Limits`) allow that many times more requests.

Your limits are shown on the **Teams** page in the Aqueduct UI, your current usage in the **API Keys** window. Need
more? See [I need more capacity for a large job](https://datalab.tuwien.ac.at/aiml/faq/#i-need-more-capacity-for-a-large-job-what-can-i-do).

### How large is the context window, and how long can answers be? { #how-large-is-the-context-window-and-what-should-i-set-as-max-output-tokens }

**As listed per model, including the output; answers can be up to 65536 tokens by default.** If answers are cut short,
check your client's own `maxTokens` setting.

Details: [Available Models: Context and output tokens](https://datalab.tuwien.ac.at/aiml/aqueduct/models/#context-and-output-tokens)
{ .faq-more }

### How do I turn thinking (reasoning) off or down? { #how-do-i-disable-thinkingreasoning-for-models }

**Set `enable_thinking` or `reasoning_effort` in `chat_template_kwargs`.** If you get
`aqueduct.UnsupportedParamsError`, your client doesn't support `reasoning_effort`.

Details: [Available Models: Reasoning](https://datalab.tuwien.ac.at/aiml/aqueduct/models/#reasoning)
{ .faq-more }

## Data and Privacy

### Where are the models hosted?

**On TU Wien hardware.** All models run on-premises, and your requests are processed there.

### What data is stored?

**Your account and usage data (e.g. model, token counts, request times); usage data is kept for about seven days.**
Prompts and answers sent through the Chat Completions API are not stored. Responses API answers are stored by default;
send `"store": false` to prevent that. Uploaded files, batches and vector stores are scheduled for deletion after at most
30 days.

Details: [Data Privacy: Retention period](https://datalab.tuwien.ac.at/aiml/privacy/#retention)
{ .faq-more }

### Can I enter personal or confidential data?

**The general TU Wien rules apply.** Personal data of others only if it's necessary and covered by a legal basis, and
never in web search.

Details: [Data Privacy: What data you may enter](https://datalab.tuwien.ac.at/aiml/privacy/#what-data-you-may-enter) and
[Terms of Service: Legal requirements](https://datalab.tuwien.ac.at/aiml/terms/#legal-requirements)
{ .faq-more }

## Help

### A model is slow, times out or seems broken. What do I do? { #why-is-a-model-not-responding-or-timing-out }

<span id="is-something-broken-where-should-i-report-it"></span><span id="is-there-a-monitoring-dashboard"></span>**Check the [monitoring dashboard](https://monitoring.ai.datalab.tuwien.ac.at/d/750918278/vllm-dashboard) first.** It
shows the current load of the inference servers and whether a model is queued, overloaded or serving normally. Models
occasionally get overloaded, which leads to long queues and timeouts.

Known issues and maintenance windows are announced in the [Matrix room](https://matrix.to/#/#llm-service:tuwien.ac.at),
often with a thread for the problem you are seeing. If there is none, open one with the model name, the request
parameters and the exact error message.

!!! question "Still have a question?"

    Ask in the [Matrix room](https://matrix.to/#/#llm-service:tuwien.ac.at). If it comes up often, we'll add it here.