baidu
/

ERNIE-4.5-300B-A47B-2Bits-TP4-Paddle

Text Generation

Model card Files Files and versions

lixc99 commited on Sep 11

Commit

ed04e1d

·

verified ·

1 Parent(s): a2289ba

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -104,7 +104,7 @@ python -m fastdeploy.entrypoints.openai.api_server \
        --port 8180 \
        --metrics-port 8181 \
        --engine-worker-queue-port 8182 \
-       --tensor-parallel-size 2 \
        --max-model-len  32768 \
        --max-num-seqs 128
 ```

        --port 8180 \
        --metrics-port 8181 \
        --engine-worker-queue-port 8182 \
+       --tensor-parallel-size 4 \
        --max-model-len  32768 \
        --max-num-seqs 128
 ```