Changelog

• 831a90d3b0f5aa0fda842e79e6f5d213b5a766cc Add different timeout scenarios to Process.checkHealthEndpoint #276 (#278)
• 977f1856bbca9b6fdf86159db1d8147b039da94f add /completion endpoint (#275)
• 52b329f7bc8124d35b21b2181c363879918e5a9b Fix #277 race condition in ProcessGroup.ProxyRequest when swap=true

Changelog

• 57803fd3aae930cc2b9a886ca034112331debada Support llama-server’s /infill endpoint (#272)
• c55d0cc8428e3bdc9b754dee0774dc413dcf1e8f Add docs for model.concurrencyLimit #263 [skip ci]

Changelog

• 7acbaf471298141d6bfe8b77444300cfe0fdde33 Add connection status indicator in UI (#260)

Changelog

• fcc5ad135aa87c85f4ae5b26fd354c8618ca63cb UI: Allow editing of title (#246)
• 305e5a0031d6adad71317f26254dd8d6f618cc79 improve example config [skip ci]

Changelog

• 04fc67354a748935e56b3660749fd9f3405eff3b Improve Activity event handling in the UI (#254)
• 4662cf7699ed8add9d46081b9049ca7c82e1bd56 add ‘unconfirmed bug’ as default label in bug-report.md

Changes:

Pre-loading of models with hooks

• Using hooks.on_startup.preload a set of models can be automatically started on startup.

# hooks: a dictionary of event triggers and actions
# - optional, default: empty dictionary
# - the only supported hook is on_startup
hooks:
  # on_startup: a dictionary of actions to perform on startup
  # - optional, default: empty dictionar
  # - the only supported action is preload
  on_startup:
        # preload: a list of model ids to load on startup
        # - optional, default: empty list
        # - model names must match keys in the models sections
        # - when preloading multiple models at once, define a group
        #   otherwise models will be loaded and swapped out
    preload:
      - "llama"

Prompt Processing Metrics added to Activities page in UI

Changelog

• 5dc6b3e6d936893fa45e98dadb6a829416f4b2d0 Add barebones but working implementation of model preload (#209, #235)
• 74c69f39ef61937ac82a6a3fdae8bb27d5e0418a Add prompt processing metrics (#250)
• a1863188924485ae242f7a6b940720744a4215ae Update Readme, Add screenshot for Activities page [skip ci]
• c4e4d5e1e953d58f6beca9781b32f5284bb8ff87 Update Readme UI Screenshot [skip ci]

Changelog

• 7985e94ba4f9c93e8d16da00cdc6f9a8da6db04b add tokens processed to ui models page
• 74556c3a36a75f9560bb76d9f42935801cec6a84 Update bug-report.md [skip ci]
• 5c381e4b3075e6e0271c5df4e343b385793049a4 Add gofmt linting to ci
• 10569ed5464ad9def7bdc722b440b78d7386e4b0 Fix model alias usage in upstream path (#230)
• 5b10b3c23f7e6258a6fced7ceba3e10a39f24c58 UI Tweaks (#228)

Changelog

• 45ea792a3aa053486d1cc77ccc79e610cdbed0b7 Fix UI panel not saving position correctly
• 1bc2802353d11c7d259c80fa1b2d5517126da2bf fix panels not saving sizing state
• 701476c0c47ed9587003b33d30a39fc19ebd187b Update README.md - remove contributor block [skip ci]
• 5c63e0066c0ab162a76e33c6591c1dfe35ada3d5 return models sorted by id in /v1/models (#222)
• 8be5073c51b2c3341151e0e8645deda06135c5c1 Fix typo (#223) [skip ci]
• 6307bd32054f718bd25398a8e20ec81363a7467e Add support for building Linux ARM64 binary in Makefile (#221)

Changelog

• 558a72de174c8a66cace12ba5d93ccbc03725f73 UI Improvements (#219)
• dc42cf366d126bccefa73205e607a2ddabff3d82 Add config monitor support for k8s configmap. (#217)
• ba0a81937ad5125d739c672606752f2d25e8191e Update README.md (#216)

Changelog

• 574fdfabb430f06e60781d628fa534ad1a8457b4 UI improvements (#213)
• 5172cb2e123b9719a1625f23915ec4e31e60b4cd Update docs in Readme [skip ci]