TextEmbed - Embedding Inference Server¶

Maintained by Keval Dekivadiya, TextEmbed is licensed under the Apache-2.0 License.

TextEmbed is a high-throughput, low-latency REST API designed for serving vector embeddings. It supports a wide range of sentence-transformer models and frameworks, making it suitable for various applications in natural language processing.

Features¶

• High Throughput & Low Latency: Designed to handle a large number of requests efficiently.
• Flexible Model Support: Works with various sentence-transformer models.
• Scalable: Easily integrates into larger systems and scales with demand.
• Batch Processing: Supports batch processing for better and faster inference.
• OpenAI Compatible REST API Endpoint: Provides an OpenAI compatible REST API endpoint.
• Single Line Command Deployment: Deploy multiple models via a single command for efficient deployment.
• Support for Embedding Formats: Supports binary, float16, and float32 embeddings formats for faster retrieval.

Getting Started¶

Prerequisites¶

Ensure you have Python 3.10 or higher installed. You will also need to install the required dependencies.

Installation via PyPI¶

Install the required dependencies:

In [ ]:

!pip install -U textembed

!pip install -U textembed

Start the TextEmbed Server¶

Start the TextEmbed server with your desired models:

In [ ]:

!python -m textembed.server --models sentence-transformers/all-MiniLM-L12-v2 --workers 4 --api-key TextEmbed

!python -m textembed.server --models sentence-transformers/all-MiniLM-L12-v2 --workers 4 --api-key TextEmbed

Example Usage with llama-index¶

Here's a simple example to get you started with llama-index:

In [ ]:

from llama_index.embeddings.textembed import TextEmbedEmbedding

# Initialize the TextEmbedEmbedding class
embed = TextEmbedEmbedding(
    model_name="sentence-transformers/all-MiniLM-L12-v2",
    base_url="http://0.0.0.0:8000/v1",
    auth_token="TextEmbed",
)

# Get embeddings for a batch of texts
embeddings = embed.get_text_embedding_batch(
    [
        "It is raining cats and dogs here!",
        "India has a diverse cultural heritage.",
    ]
)

print(embeddings)

from llama_index.embeddings.textembed import TextEmbedEmbedding # Initialize the TextEmbedEmbedding class embed = TextEmbedEmbedding( model_name="sentence-transformers/all-MiniLM-L12-v2", base_url="http://0.0.0.0:8000/v1", auth_token="TextEmbed", ) # Get embeddings for a batch of texts embeddings = embed.get_text_embedding_batch( [ "It is raining cats and dogs here!", "India has a diverse cultural heritage.", ] ) print(embeddings)

[[0.07680495083332062, -0.05040504038333893, 0.11367770284414291, 0.05823838338255882, 0.022227859124541283, -0.04733077064156532, 0.05152371898293495, 0.05070779100060463, 0.0006111942930147052, 0.047624967992305756, -0.051811832934617996, -0.07829486578702927, 0.04556303843855858, -0.05040515959262848, 0.0029875831678509712, -0.011835750192403793, -0.0026014496106654406, -0.08340940624475479, 0.006381386425346136, -0.08341386169195175, -0.06011611223220825, -0.036668118089437485, -0.07939755916595459, -0.01343041192740202, -0.06547989696264267, 0.12642565369606018, -0.06321138143539429, 0.05382829159498215, -0.052164286375045776, -0.00024023318837862462, -0.015094783157110214, -0.05263998731970787, 0.00974583625793457, 0.007128292229026556, -0.054314471781253815, 0.01022003311663866, 0.12224774807691574, -0.0658874586224556, 0.012737945653498173, 0.05347715690732002, 0.03169456124305725, -0.05292658135294914, -0.042729008942842484, 0.03916006162762642, 9.223353845300153e-05, 0.059408292174339294, 0.020869968459010124, -0.0401160828769207, 0.0027611232362687588, 0.004130088724195957, 0.0540916845202446, 0.06629195064306259, -0.006132130045443773, -0.006394094787538052, -0.03519970551133156, -0.07069820165634155, 0.011002703569829464, 0.02455390989780426, 0.00501644192263484, 0.05660104751586914, -0.006037208717316389, 0.08306437730789185, 0.007119494024664164, 0.08410854637622833, -0.01210033055394888, 0.01966363564133644, 0.012595041655004025, 0.03973742201924324, 0.009766259230673313, -0.06565872579813004, -0.04130473732948303, 0.0624854601919651, 0.045739103108644485, -0.06709646433591843, -0.04065104201436043, 0.04857759550213814, 0.06335414201021194, 0.009452641941606998, 0.03526662662625313, -0.060091376304626465, 0.03666703402996063, -0.09009642153978348, 0.024401679635047913, -0.03236083686351776, 0.01864292472600937, 0.08615810424089432, -0.02484048902988434, 0.053208936005830765, -0.04861944168806076, -0.04714304581284523, -0.00996045395731926, -0.04190995916724205, 0.00621739961206913, -0.025682389736175537, -0.048724569380283356, 0.048258986324071884, 0.09742002189159393, -0.04164482653141022, -0.0632825717329979, 0.10244321823120117, 0.008165179751813412, -0.031598374247550964, 0.033554621040821075, 0.015209853649139404, 0.004974573850631714, -0.00926778931170702, 0.02616698294878006, -0.024738963693380356, 0.0626964271068573, -0.033648233860731125, 0.019024012610316277, -0.04049520567059517, -0.026059657335281372, 0.03538138046860695, 0.013703618198633194, 0.011509755626320839, 0.014014758169651031, 0.06699714064598083, 0.017573079094290733, 0.006561113055795431, -0.04278159141540527, 0.10309535264968872, -0.0463445670902729, -0.008647579699754715, 0.08202703297138214, -0.03999117761850357, -0.06685960292816162, -0.006554177030920982, 0.04223375394940376, -0.0012740814127027988, 0.1117301657795906, 0.0017128527397289872, 0.05188003554940224, -0.12444354593753815, -0.009806379675865173, 0.015819454565644264, 0.006935478653758764, -0.023190373554825783, 0.02546553499996662, -0.013335909694433212, 0.015678856521844864, 0.011434794403612614, 0.021772846579551697, -0.016049562022089958, 0.06551887840032578, -0.033432554453611374, -0.05978545919060707, 0.02994738332927227, 0.0312717966735363, -0.029713870957493782, 0.05580022558569908, -0.0029229489155113697, -0.015401207841932774, -0.08176562190055847, 0.00724291754886508, -0.012905357405543327, 0.06373728811740875, 0.044775426387786865, 0.042971812188625336, 0.019208597019314766, 0.007659510709345341, 0.05636291950941086, -0.07192669063806534, -0.007240221370011568, -0.12496685981750488, -0.09168796986341476, -0.01385537814348936, -0.01781819388270378, -0.003153262659907341, -0.03902065008878708, -0.01590655744075775, -0.07812319695949554, 0.08514761179685593, 0.05925028771162033, -0.08532647788524628, -0.0069305249489843845, -0.10550417006015778, 0.0030866223387420177, 0.03278058022260666, 0.0771193727850914, 0.025492709130048752, 0.07974281907081604, -0.0247025229036808, 0.0017616583500057459, 0.007897263392806053, -0.02404951862990856, -0.07377568632364273, 0.018059518188238144, 0.06308440864086151, 0.007412976585328579, 0.12542572617530823, -0.05442231893539429, 0.032104313373565674, -0.08016591519117355, -0.017097385600209236, 0.09793233126401901, 0.013995948247611523, 0.06925566494464874, 0.01833397150039673, 0.07771677523851395, 0.015293139964342117, -0.08090431988239288, 0.05007411539554596, -0.09761709719896317, 0.00423071626573801, -0.10888469219207764, 0.08356834203004837, 0.05643865466117859, -0.06505356729030609, 0.028523283079266548, 0.041734449565410614, -0.04735998436808586, 0.0603979118168354, 0.00788224209100008, 0.0059880749322474, 0.016858983784914017, -0.005104954820126295, -0.014090330339968204, -0.016895350068807602, 0.07090745866298676, 0.027565861120820045, -0.07306291162967682, 0.011732944287359715, 1.3316887504349744e-32, 0.0641976147890091, -0.06777289509773254, -0.04849627986550331, -0.061756569892168045, -0.00045659070019610226, 0.05062531679868698, -0.02032887563109398, -0.00010833075066329911, -0.034192439168691635, 0.09903416037559509, -0.09775568544864655, 0.019806770607829094, 0.01679135672748089, 0.04494625702500343, -0.005360170733183622, 0.03474418818950653, -0.010824315249919891, 0.012423712760210037, -0.04247642681002617, 0.0883023589849472, 0.0054736025631427765, 0.010927718132734299, -0.08288439363241196, 0.08409574627876282, -0.049664005637168884, 0.03450685366988182, 0.0549996979534626, 0.026557954028248787, -0.005721907131373882, -0.044021379202604294, -0.012303856201469898, -0.05126556009054184, -0.06447187066078186, 0.004302806220948696, 0.009201552718877792, 0.030752968043088913, 0.04058847576379776, -0.10559576004743576, -0.05059872940182686, 0.03609590604901314, -0.02307843416929245, -0.09876292198896408, 0.1177290678024292, 0.04885876923799515, -0.030000679194927216, 0.029548928141593933, 0.00706587266176939, -0.02042660303413868, -0.039021506905555725, 0.04144273325800896, 0.00988572183996439, -0.11107584089040756, -0.08385508507490158, 0.005822864826768637, 0.0012109441449865699, 0.004146373365074396, -0.031179670244455338, -0.04467696323990822, 0.015674574300646782, 0.1341249942779541, -0.07928548753261566, 0.013704975135624409, 0.05670353025197983, 0.060587864369153976, 0.013574873097240925, -0.10535777360200882, -0.007025950122624636, -0.023063747212290764, 0.06690023094415665, -0.06298024207353592, -0.005376291461288929, 0.04811660945415497, -0.13402746617794037, 0.007578582037240267, -0.0024443611036986113, -0.06605108827352524, 0.023479076102375984, 0.026372628286480904, 0.02109520137310028, 0.061991095542907715, -0.023896468803286552, 0.04404592141509056, -0.008029282093048096, 0.020084016025066376, 0.014552260749042034, -0.0643354281783104, 0.08181426674127579, 0.0355604849755764, 0.06744600087404251, -0.025436848402023315, 0.012248354032635689, 0.04581432789564133, 0.060171257704496384, -0.062297578901052475, 0.004475506022572517, 1.1358202409876845e-32, -0.003122281515970826, -0.020873863250017166, -0.08247245848178864, -0.03895285725593567, 0.06328216195106506, -0.008792975917458534, 0.010481133125722408, 0.006121897138655186, -0.043270811438560486, -0.048157405108213425, 0.07098594307899475, 0.002934532705694437, 0.03133571520447731, 0.04665524140000343, 0.07324907183647156, 0.011480768211185932, 0.043815914541482925, -0.03362172469496727, 0.05287368968129158, -0.009842237457633018, -0.022900082170963287, 0.06757314503192902, -0.11671338230371475, 0.02805415727198124, 0.04499270021915436, 0.021897515282034874, 0.0017030314775183797, 0.004345265217125416, 0.03667227923870087, 0.0067671057768166065, -0.020933877676725388, 0.027675405144691467, -0.08036870509386063, -0.09826663136482239, -0.07401884347200394, -0.02790938690304756, 0.057265762239694595, -0.058603350073099136, 0.04551159590482712, -0.06825559586286545, 0.09759797900915146, -0.04156721010804176, 0.013710612431168556, -0.06947797536849976, -0.03834575042128563, -0.023449581116437912, 0.034450747072696686, 0.017096437513828278, 0.007188005838543177, -0.018480243161320686, -0.07647694647312164, 0.019606871530413628, 0.09614947438240051, -0.01545009110122919, -0.008134815841913223, 0.026222800835967064, 0.03478577733039856, -0.046021610498428345, -0.003438845742493868, 0.0031483143102377653, -0.04465096443891525, -0.013187187723815441, -0.03098474256694317, 0.06542713940143585], [0.07799432426691055, 0.07165352255105972, 0.020055454224348068, -0.06940069049596786, -0.10845369100570679, 0.013620556332170963, 0.05485392361879349, -0.07176253944635391, -0.02918390929698944, -0.00823366828262806, -0.003418993204832077, -0.09665513783693314, -0.011166798882186413, 0.05481532961130142, 0.019631581380963326, 0.05331071466207504, 0.021168764680624008, 0.0026415239553898573, 0.001307831727899611, -0.08274262398481369, 0.05891267582774162, 0.049844369292259216, 0.02702411822974682, 0.01655544526875019, -0.02096525765955448, -0.03273285925388336, 0.04488828033208847, 0.038071081042289734, 0.044224970042705536, -0.05361603945493698, 0.034158945083618164, 0.0718199610710144, -0.018762726336717606, -0.02206559106707573, -0.05857796594500542, -0.0635826364159584, -0.030340351164340973, 0.04317975416779518, 0.12967708706855774, 0.05035289376974106, -0.051993947476148605, 0.07054612785577774, 0.10430995374917984, 0.022495536133646965, -0.0518866702914238, 0.08801426738500595, 0.05253857374191284, 0.043512847274541855, -0.055786993354558945, 0.03341323137283325, 0.005513152107596397, 0.005101369693875313, -0.0672992542386055, -0.020335936918854713, 0.016071878373622894, 0.018543681129813194, -0.035623613744974136, 0.04357553645968437, 0.029248248785734177, 0.08883709460496902, 0.020363351330161095, -0.02196011319756508, 0.0012850118800997734, 0.006192635744810104, -0.014710276387631893, -0.015054947696626186, 0.10157758742570877, 0.09813037514686584, 0.025180503726005554, -0.06945953518152237, -0.015296557918190956, 0.06923636049032211, 0.07874289155006409, 0.04599646478891373, 0.05999474599957466, -0.07447455078363419, -0.006511614192277193, -0.010188503190875053, -0.17496728897094727, 0.005023193079978228, 0.032461415976285934, 0.0930548906326294, 0.0271072406321764, 0.01091151125729084, 0.01478418055921793, -0.015351216308772564, 0.028111444786190987, 0.01782655529677868, -0.00426711468026042, 0.003396136686205864, -0.02513243816792965, 0.05175793170928955, 0.07105625420808792, 0.013885803520679474, 0.02295723743736744, -0.003022288903594017, 0.05607959255576134, -0.03689858317375183, 0.06573966145515442, 0.06234217807650566, -0.030095433816313744, 0.031915076076984406, -0.01025893073529005, 0.043908096849918365, -0.0758497416973114, -0.033421557396650314, -0.07022690773010254, -0.04092619940638542, -0.002199685899540782, -0.030839109793305397, -0.035326551645994186, 0.05153883621096611, -0.059227559715509415, 0.05292855575680733, 0.02773517370223999, 0.011323811486363411, 0.0034195969346910715, -0.07174813747406006, 0.015532704070210457, -0.02600107528269291, -0.08129710704088211, 0.002865569433197379, 0.0894491970539093, -0.029829584062099457, 0.026074862107634544, -0.012456013821065426, -0.041887104511260986, -0.004592636600136757, -0.0032614741940051317, -0.011445867829024792, 0.017015833407640457, 0.026977282017469406, -0.0029720801394432783, -0.031186817213892937, -0.02013154700398445, -0.0189276821911335, -0.010720884427428246, -0.13561150431632996, 0.07252978533506393, 0.039279550313949585, 0.00860431045293808, 0.002585938200354576, 0.020295260474085808, -0.005369565449655056, -0.09946524351835251, 0.04092240333557129, 0.05004318431019783, 0.053540512919425964, -0.0011289584217593074, 0.014238565228879452, 0.02057207003235817, 0.07597292214632034, 0.022981945425271988, 0.02112702652812004, 0.055218324065208435, 0.017567919567227364, -0.03778007626533508, 0.029483158141374588, 0.022342726588249207, 0.0006534780259244144, 0.02237580344080925, -0.03161503002047539, 0.044973742216825485, 0.05495552718639374, 0.06669220328330994, -0.04954477772116661, -0.03808807209134102, -0.02870265021920204, -0.004877750761806965, 0.0019964200910180807, -0.0963866338133812, 0.01768258959054947, 0.010873222723603249, 0.04940221831202507, 0.05496761202812195, -0.037347279489040375, -0.09413357824087143, 0.013636430725455284, -0.07790953665971756, 0.020940089598298073, -0.02058587037026882, -0.026977892965078354, -0.031237175688147545, 0.021282032132148743, 0.01815761812031269, 0.01060271356254816, 0.013289585709571838, 0.0031410097144544125, -0.03254946321249008, -0.02277086302638054, 0.05981164053082466, 0.007833489216864109, 0.021829526871442795, 0.10879793763160706, 0.07313315570354462, -0.0029729963280260563, -0.034609604626894, -0.012553699314594269, -0.01456522662192583, 0.05413827300071716, -0.023727990686893463, 0.01579025201499462, -0.03655393794178963, 0.0019323736196383834, 0.04180413484573364, -0.005496712401509285, 0.06435801833868027, 0.0364333800971508, -0.06671112775802612, 0.07372396439313889, -0.01814170554280281, -0.025395965203642845, -0.0047453115694224834, -0.03013179823756218, -0.03588418662548065, -0.09724976122379303, 0.05876237899065018, 0.007222804240882397, 0.07826775312423706, -0.023003585636615753, 0.0003603142686188221, -0.08320706337690353, 0.006020791828632355, 1.1430100787160921e-32, 0.03946039825677872, 0.021631676703691483, -0.011926254257559776, 0.07426220923662186, -0.04159925505518913, -0.045727528631687164, -0.05295009911060333, 0.0067450376227498055, -0.012675588019192219, 0.004204368218779564, 0.03162655606865883, 0.04263368621468544, 0.11720915883779526, 0.044679418206214905, -0.05283312872052193, 0.003912652377039194, 0.0249093696475029, 0.09611153602600098, -0.025266725569963455, 0.08444087952375412, 0.06580599397420883, 0.04309471324086189, -0.008736539632081985, -0.014878572896122932, -0.08842268586158752, 0.041510771960020065, -0.14692062139511108, -0.03553466498851776, -0.018853971734642982, -0.05361965298652649, 0.14357304573059082, -0.07812920957803726, -0.04741043969988823, -0.05659728869795799, -0.038966599851846695, -0.025837620720267296, -0.011006047017872334, -0.02772398851811886, 0.09088914841413498, 0.08837723731994629, -0.07656233757734299, 0.06411263346672058, 0.004343628417700529, 0.08005384355783463, -0.00722179189324379, 0.003935917280614376, -0.022400356829166412, 0.03254381939768791, -0.024593714624643326, -0.01973755657672882, -0.014805461280047894, -0.05651266872882843, 0.037547145038843155, -0.11008766293525696, 0.08978398889303207, -0.03053450584411621, 0.004431360401213169, 0.016068505123257637, -0.022344455122947693, -0.07138761878013611, 0.01603115163743496, -0.006439351011067629, -0.010920426808297634, -0.03738963603973389, -0.016587689518928528, -0.03227771446108818, 0.17045475542545319, 0.004980448167771101, -0.02731807343661785, 0.06359931081533432, 0.03456397354602814, 0.007073391694575548, -0.12932629883289337, -0.004573840647935867, -0.005181489512324333, 0.040059011429548264, 0.06406287103891373, 0.030044276267290115, 0.09301282465457916, 0.09408746659755707, -0.01744355820119381, -0.004212620668113232, 0.011645005084574223, -0.02169305458664894, 0.0563432052731514, 0.09795628488063812, -0.073667012155056, -0.013222085312008858, 0.07637935131788254, -0.06758248060941696, 0.001244493294507265, 0.027596281841397285, -0.0487433597445488, -0.036038145422935486, 0.060777854174375534, 3.8837697849534966e-33, -0.05240064114332199, -0.021643461659550667, -0.04175921529531479, -0.06045977771282196, -0.06952421367168427, -0.038215044885873795, 0.051367416977882385, 0.012103230692446232, 0.046106696128845215, 0.04173969477415085, -0.08288600295782089, -0.02326829358935356, -0.012264708057045937, 0.012436570599675179, 0.013261312618851662, 0.05146365240216255, -0.04755478352308273, 0.05892452225089073, 0.018805019557476044, -0.018536118790507317, 0.04131010174751282, -0.09599348157644272, 0.050808556377887726, -0.027801524847745895, -0.03190303221344948, 0.019173959270119667, -0.09970079362392426, -0.08847703039646149, -0.007989868521690369, -0.029317224398255348, -0.06539823114871979, 0.05633286014199257, -0.03425817936658859, -0.09287241846323013, 0.0006628779228776693, -0.03786216303706169, 0.06579028069972992, -0.09779658913612366, 0.0270419679582119, -0.04114564135670662, 0.08666373789310455, -0.07169074565172195, -0.014005975797772408, 0.016654517501592636, 0.006269460078328848, -0.04980370029807091, -0.050808873027563095, 0.014376435428857803, -0.05227877199649811, -0.07605838775634766, -0.037085626274347305, -0.03101062960922718, -0.014809953048825264, 0.030327126383781433, -0.06510411202907562, -0.022612834349274635, -0.07160919159650803, 0.004284658469259739, 0.012747908011078835, -0.014764118008315563, 0.07312114536762238, -0.07069794833660126, 0.002908692229539156, -0.044137585908174515]]

For more information, please read the documentation.