Success Story: RISE Accesses VEGA

Organisations & Codes Involved:

Research Institutes of Sweden, RISE

Technical/scientific Challenge:

RISE NLU Group will train English BERT model using multiple GPUs on the EuroHPC JU system Vega.

With the help of ENCCS, Evangelia Gogoulou from RISE NLU Group, Digital Systems got access to the EuroHPC JU system Vega at Izum, Slovenia. She will be training from scratch an English BERT model using multiple GPUs and then evaluate its downstream performance on the GLUE benchmark (https://gluebenchmark.com/ ).

Solution:

The model performance will be compared with the English BERT trained on one GPU. The second task is to start from the English BERT model and continue pre-training it on Russian, on multiple GPUs. The transferred Russian model will be then evaluated on GLUE and the results will be compared with existing Russian Language models.

SUCCESS STORY # HIGHLIGHTS:

Keywords: High-Performance Computing, HPC, Supercomputing, Engineering, Software Optimisation, EuroCC
Industry sector: IT/HPC systems, services & software providers, manufacturing & engineering, natural science
Technology: HPC, HPDA, AI

Contact:

Apostolos Vasileiadis apostolos.vasileiadis@it.uu.se (EuroCC Sweden – National Competence Centre)

This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 951732. The JU receives support from the European Union’s Horizon 2020 research and innovation program and Germany, Bulgaria, Austria, Croatia, Cyprus, the Czech Republic, Denmark, Estonia, Finland, Greece, Hungary, Ireland, Italy, Lithuania, Latvia, Poland, Portugal, Romania, Slovenia, Spain, Sweden, the United Kingdom, France, the Netherlands, Belgium, Luxembourg, Slovakia, Norway, Switzerland, Turkey, Republic of North Macedonia, Iceland, Montenegro