si `ddlZddlZddlZddlmZGddZGddeZdS)N) InputExamplec<eZdZdZddddejdddfdZd d Zd S) STSDataReadera1Reads in the STS dataset. Each line contains two sentences (s1_col_idx, s2_col_idx) and one label (score_col_idx) Default values expects a tab separated file with the first & second column the sentence pair and third column the score (0...1). Default config normalizes scores from 0...5 to 0...1 rr Tc ||_||_||_||_||_||_||_||_| |_dS)N) dataset_folder score_col_idx s1_col_idx s2_col_idx delimiterquotingnormalize_scores min_score max_score) selfr r rr rrrrrs ^/var/www/icac/venv/lib/python3.11/site-packages/sentence_transformers/readers/STSDataReader.py__init__zSTSDataReader.__init__sJ-*$$" 0""c tj|j|}|drt j|ddnt |d5}tj||j |j }g}t|D]\}}t||j } |jr| |jz |j|jz z } ||j} ||j} |t)|t+|z| | g| |dkrt-||krnd d d n #1swxYwY|S) zJfilename specified which data split to use (train.csv, dev.csv, test.csv).z.gzrtutf8)encodingzutf-8)rr)guidtextslabelrN)ospathjoinr endswithgzipopencsvreaderrr enumeratefloatr rrrr rappendrstrlen) rfilename max_examplesfilepathfIndataexamplesidrowscores1s2s r get_exampleszSTSDataReader.get_examples$s7<< 3X>>;C;L;LU;S;S TYx 7 7 7 7Y] wZ Z Z   :cT^T\RRRDH$T??  Cc$"4566(Y"T^38WXE)) (SWW2DRQSH\a b b bccc!##H (E(EE               "s#C EEEN)r)__name__ __module__ __qualname____doc__r% QUOTE_NONErr7rrrrsb####,rrc>eZdZdZddddejdddffd ZxZS) STSBenchmarkDataReaderzReader especially for the STS benchmark dataset. There, the sentences are in column 5 and 6, the score is in column 4. Scores are normalized from 0...5 to 0...1 r rTrc \t|||||||||  dS)N) r r rr rrrrr)superr) rr r rr rrrrr __class__s rrzSTSBenchmarkDataReader.__init__@sK )!!'-  r)r8r9r:r;r%r<r __classcell__)rDs@rr?r?;sc          rr?)r%r#rrrr?r=rrrGs 00000000f     ]     r