Tm server to use with omegat
Thread poster: Руслан И.
Руслан И.
Руслан И.
Russian Federation
Local time: 18:38
Turkish to Russian
+ ...
Dec 6, 2018

I have a linux server, I want to upload my TMs to server so that I can query translation strings from omegaT. Is there such webbased software where I can store my TMs and query them. Is there an instruction to do this? Maybe there is another way to manage large TMs?

 
Didier Briel
Didier Briel  Identity Verified
France
Local time: 14:38
English to French
+ ...
You can use team projects Dec 6, 2018

Руслан И. wrote:
I have a linux server, I want to upload my TMs to server so that I can query translation strings from omegaT. Is there such webbased software where I can store my TMs and query them. Is there an instruction to do this? Maybe there is another way to manage large TMs?

Team projects work differently (they replicate the TMs locally using version control software, such as Subversion or Git), but they provide more or less the same thing.

See for instance:
https://www.proz.com/forum/omegat_support/240607-introduction_to_the_team_project_feature_in_omegat.html

Didier


 
Руслан И.
Руслан И.
Russian Federation
Local time: 18:38
Turkish to Russian
+ ...
TOPIC STARTER
My main goal is different Dec 6, 2018

My main goal is to upload my TMs to some web platform, so that they are stored in database. This, in my opinion, will speed up large TM lookup.

 
Didier Briel
Didier Briel  Identity Verified
France
Local time: 14:38
English to French
+ ...
A database cannot be faster than memory Dec 7, 2018

Руслан И. wrote:
My main goal is to upload my TMs to some web platform, so that they are stored in database. This, in my opinion, will speed up large TM lookup.

Of course, not having to load the TM in memory is initially faster but, for the lookup itself, a database lookup cannot be faster than searching in memory.

Didier


 
Руслан И.
Руслан И.
Russian Federation
Local time: 18:38
Turkish to Russian
+ ...
TOPIC STARTER
TM size is a much bigger than RAM size Dec 7, 2018

Didier Briel wrote:
a database lookup cannot be faster than searching in memory.


Yes, I agree, but total size of my TMs is much bigger than avaliable memory size on my PC.


 
tcordonniery
tcordonniery
France
Local time: 14:38
Server TM? Jan 22, 2019

Hi Ruslan

Following my answer to your previous question:
as I said, actually this not possible to write a plugin for external translation memories, because OmegaT does not provide the correct API.

But I also experimented the possibility to write such an API. The Lucene indexes I mentioned is only one of the possible implementations: access to a server is another potential plugin using the same API.

Yo
... See more
Hi Ruslan

Following my answer to your previous question:
as I said, actually this not possible to write a plugin for external translation memories, because OmegaT does not provide the correct API.

But I also experimented the possibility to write such an API. The Lucene indexes I mentioned is only one of the possible implementations: access to a server is another potential plugin using the same API.

You can experiment my project Cyclotis - http://www.silvestris-lab.org/node/12 - which uses a Postgresql database for this. But actually Cyclotis is designed for very small memories in real time (i.e. for ongoing segments, so that your colleages see immediately what you are translating). The project for a bigger database, which is possible but would require some architectural changes, is mentioned in my site but is not really started now.

Didier Briel wrote:

Team projects work differently (they replicate the TMs locally using version control software, such as Subversion or Git), but they provide more or less the same thing.


No, they do not provide the same thing as a client/server memory: Team projects are stored in a server but you have to checkout the entire project and then TMX are loaded in memory. OmegaT never does queries to a server for translation memories (only for machine translations)

Didier Briel wrote:

for the lookup itself, a database lookup cannot be faster than searching in memory.



That is not true, because databases are usually indexed: when you do a lookup, they do not explore all the entries, which would be a linear algorithm, they look in the index itself, so the algorithm becomes logarithmic. That is how databases can be faster for any kind of search, not only for translation memories.




[Edited at 2019-01-22 07:12 GMT]

[Edited at 2019-01-22 07:13 GMT]
Collapse


 
richk
richk
Switzerland
Large TM files in Team Projects can cause version control performance issues Jul 8, 2019

Moving the TM data to a sever / database would also allow to avoid the performance issues caused by the large and frequently updated project TM file in a Team Project.

I only have experiences with Git which already starts struggling with a 10-15MB TM file. Git is not able to efficiently and quickly compress the changes in the big TM file which makes the repository to grow quickly to hundreds of MBs or even GBs in size. Git's delta compression is not used initially in this case and w
... See more
Moving the TM data to a sever / database would also allow to avoid the performance issues caused by the large and frequently updated project TM file in a Team Project.

I only have experiences with Git which already starts struggling with a 10-15MB TM file. Git is not able to efficiently and quickly compress the changes in the big TM file which makes the repository to grow quickly to hundreds of MBs or even GBs in size. Git's delta compression is not used initially in this case and with each commit of the project's TM the repo grows by the zlib compressed size of the TM file.

Only aggressive Git garbage collection can reduce the size of these repos effectively, but it takes a lot of CPU, RAM and time to complete it successfully. Even then it is temporally and the repo starts growing quickly again.

One way to workaround this would be storing the TM on a server. This way the TM's of other projects could also be made available much easier. The down site of an OmegaT TM server would be the dependency on the TM server. Currently, we can continue working locally if the network is down.
Collapse


 


There is no moderator assigned specifically to this forum.
To report site rules violations or get help, please contact site staff »


Tm server to use with omegat






Anycount & Translation Office 3000
Translation Office 3000

Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.

More info »
Trados Studio 2022 Freelance
The leading translation software used by over 270,000 translators.

Designed with your feedback in mind, Trados Studio 2022 delivers an unrivalled, powerful desktop and cloud solution, empowering you to work in the most efficient and cost-effective way.

More info »