Unicode Font

International language support is perhaps the feature I'm most excited about in 1.6. And I got it working! It's great.

However, it required that I install the Arial Unicode font, which I had to find online: http://prdownloads.sourceforge.net/vietunicode/Aruniupd.exe?download (thanks to Kex and #secondlife for the pointer). Unfortunately, Microsoft no longer distributes this except with some of its programs (not Windows).

Without this font, the Unicode extended characters are displayed as block-shaped characters.

My request is that, if possible, a Unicode font be distributed with SL that will work with it, so that everyone can view Unicode characters rather than block-shaped characters (without any additional work). As there would probably be licensing issues with distributing Microsoft's Arial Unicode, maybe a freely distributable font could be included with SL.

As an alternative, or in addition, perhaps a font selection dialog box could be included under Preferences that would allow users to select a font.

Anyway, great work on this and the other features.

And on the Mac, it uses some other font (not Arial), which doesn't support many languages. I can see my Chinese and Japanese, and I'm very happy about that, but I can't read Korean or a variety of other languages. I just get little squares. It would help to be able to manually set the font to one that supports the language of interest.

Also, the doublebyte characters wouldn't show at all on Medium sized chat font, which I understand is a known bug. They would still show in Chat history, though.

Another user, on XP, reported that she couldn't see what she was typing before it appeared in the chat. Kind of frustrating, especially with systems where you type phonetics and then pick the right character from the list. This might be due to funky MS stuff, though, and not directly fixable by LL.

I was a bit disappointed that one can't enter non-Latin characters in notecards, profile, or other windows. Presumably this means I wouldn't be able to script floating text in Chinese? Or name objects in hirigana?

Overall, loads of fun, but not quite ready for prime time yet.

neko
( 猫 )

What? No （；´Д｀）ノ on my profile STILL??

:\

:\

:\

It'd be nice to allow non-Roman support for things like Profile and floating text..

Oh, and using option-key combinations on the Mac is still kinda buggy. In particular, option-g generates the copyright symbol ©, which works, but which also generates an error saying you're being reported for abuse.

neko

I entered the notecard/script problem as a bug.

I was able to have a script echo text it heard via listen(). If there was a script function to write notecard lines, then a script could be written to echo lines heard via listen() into notecards, which would be a suitable workaround for the purposes of writing scripts, if the UI can't be easily fixed. (Although the read/write functions would have to support Unicode...)

The ability to write to notecard lines would be nice to see anyway.

The font should be installed by the SL installer, plain and simple. The font isn't even distributed with Windows XP.

The font is quite hefty, being approx. 13 meg I believe -- it would near-double the size of the SL installer. Also, there might be licensing and distribution issues to consider, as well as those who would only be interesting in chatting in English (or can only understand that language) anyway.

But there definitely needs to be easy access to download the Arial Unicode MS (or equivalent font) installer.

Moleculor, I absolutly agree that if the font is required but tends to be "Exotic" on systems, then it should be added to the installer package - but for the sake of argument, there's one point to concider: The thing is a whopping 22MB uncompressed. o.O I have not tried to compress it or saw it compressed yet, but the thing could definetly be a big burdain on the size of the installer file.

Perhaps the solution would be for the client to do the following thing:
- Check OS.
- If Windows, check for arieluni.ttf availability.
- If non available, open a dialog-box during the Login procedure and note that International language support requires arieluni.ttf. Click here to download the file from a mirror on SL's site.

How's that?

CrystalShard, your solutions are elegant. I think it's a good idea to have an option during the SL install sequence to ask the question, along the lines of what you are saying:



or somesuch question, and go from there. Hopefully in time your concerns about BiDi will be addressed too.

I wish to clarify that the size of the compressed font in its installer is approx 13 meg -- 22 meg is the unstuffed result, as CS said.

The size of the font is definitely an issue. Torley's idea, "Do you want international language support?" sounds great... maybe that same box could explain what that involves (having a Unicode font), and either provide some links or auto-download a suitable font.

There may be licensing issues with linking directly to a non-free font, though, which is one reason it makes sense to have a font-selection box and/or link to freely distributable Unicode fonts, which do seem to exist.

I think Neko has uncovered the true reason that LL are not distributing the arieluni.ttf



Copyright abuse !!! ROFLMAO

Wow, I was unexpectedly subtle in my comprehension of the Ways of the Lindens!

I think the font download should be an option at the time of download, not at the time of installation (though with the current setup, those can be the same thing). I think it should be a font that LL pays the license for, and should be supplied for both Mac and PC so everyone has the same one -- there are things in the MS Unicode Arial that aren't on my Mac, and vice versa. AND I think if you want to use some other unicode font on your system you ought to be able to set the font yourself, which would avoid the whole silly problem for those of us who have plenty of language-specific Unicode fonts already.

Now, a question: howsabout if I create a whole bunch of gestures with latin keyboard shortcuts which generate Chinese text + pinyin pronunciation guides and English translations, along with recorded audio of the Chinese phrase, and package them as "Chinese Lessons for your Av." Is there a market for this? (Or Japanese, for that matter?) This saves a lot of people from having to set up and learn input methods. Including the romanization (pinyin) and English translation helps keep them from being too annoying to everyone else. (And based on some research I've done, using such kits actually does help the user learn a little bit of the language.) Comments?

neko

I had no problem with either Bulgarian or Russian Cyrillic fonts in the chat, though they do disable the chat bubbles temporarily. I'm trying some other things today if everything is up. This international support is very exciting as I have a number of non-latin reading friends who are very much interested in Second Life.

I look forward to in-game Japanese tuturials

--Jack Lambert

Did anyone manage to copy and paste unicode encoded text say from mozilla into the client. I tried and got ? or a dot every time. I bug reported it, but it could well be me.

The UK keyboard works wondefully and make me very happy - so *Thank you* (gives massive hug to whoever coded that).

Looking through the list at http://www.alanwood.net/unicode/fonts.html one font caught my attention: "Code2000 – 50,710 characters (62,981 glyphs) in version 1.14 - Includes many characters that are difficult to find elsewhere, making it a useful font to assign to the user-defined encoding or character set in your Web browser, and well worth the $5 registration." This shareware font is available from http://home.att.net/~jameskass/ and the download is only 3.6MB. Perhaps some sort of licensing deal could be worked out with the creator and it could be included with SL after all. Or maybe there are other fonts that would allow an inexpensive licensing deal for SL. (As for Macs: can they not use TrueType fonts? Can anyone suggest other fonts that are complete enough and free, shareware, or otherwise cheap?)

Short of including such a font in the SL client, though, I do like the idea of having a font download link or links included alongside the link to download the SL client.

I think using SL for language instruction has _huge_ potential. This is something I very much intend to participate in, both as instructor and learner. As SL reaches a more international audience, this opportunity increases as I think there will be more demand generated for people to learn at least a bit of each other's native languages, or go beyond that to develop proficiency. I know only small bits of languages other than English, but I've done a bit of tentative English tutoring online already, and SL would make possible visual cues that are hard to present in text chat or over voice-only interaction.

As quality language instruction products and services are developed on SL, I can see this as an application that could draw a lot of people into SL, and potentially bring quite a bit of money into the SL economy. (Language instruction, in many forms, is a large and growing industry that becomes relevant to more and more people as the world becomes more interconnected.)

I'd like to add that Unicode has tons of symbols that are relevant to everyone, including English-only speakers, that could be a great extension to the output scripts can generate. I see this alone as being a reason to try to get all SL users access to a Unicode font, and put some work into getting Scripts, Notecards, and other text fields in SL the ability to handle Unicode.

Also, I thought of a work-around for the current inability to include Unicode in scripts: it may be possible to use XML-RPC to send Unicode strings to scripts from an external input source. However, the inability to send XML-RPC requests from within SL means that this would have to be done manually (by triggering the external script) at the time the SL script is initiated or used.

Comments or ideas on any of this?

Don't know about the XML-RPC workaround.

But I'm psyched about developing language lessons in SL. Two other thoughts:

1 - Since we don't have streaming audio from av yet, a live instructor can't help correct actual student pronunciation, and all the interaction will be in text with pre-recorded audio files. This doesn't make the best use of an actual capable language instructor, so it would be beneficial to be able to script NPC instructors (see NPC thread in Feature Suggestions).

2 - We need built-in streaming audio per av so a live instructor can help correct actual student pronunciation.

(Yes, I know there are 3d party products, I will need to look into them.)

3d thought -- someday it would be nice to be able to capture the user's audio, send it out for processing, and offer back a computational analysis of pronunciation, with guidance. It can be done outside of SL -- it would be very, very cool to be able to integrate this with SL. The Dept of Defense uses Unreal for this today, for Arabic lessons. SL would be much more appropriate for many purposes, and much easier to do open-source work in.

neko

Very interesting: I found an NPR program about the Unreal Arabic technology at http://www.npr.org/templates/story/story.php?storyId=4503426

NPCs would be of great use for making something similar to what's been done with Unreal.

As for live voice interaction, I've used Skype for that and it works well.

Most of the (very limited) language learning and instruction I've done has been on text chat, which is useful for teaching spelling and grammar and introducing vocabulary. I'm in the middle of an online English instruction certificate program, actually - although I haven't done any work on it since I started using SL(!) The prospect of integrating language instruction into my other ideas for using SL for learning and instruction will likely get me going on that soon, though.

The Pimsleur CD programs are supposed to work pretty well without any instructor feedback, and in fact without much of any text or images either. (Although the lack of interactivity can be a motivational problem.) I've used a program called Rosetta Stone that has a simple automatic voice recognition feature for automatic pronunciation feedback (one word or phrase at a time): http://www.rosettastone.com/

So, there are a number of different things that could be developed using SL that could rival other language instruction products.

Make the client test for the existence of the font. If it doesn't exist, have the client download the font as a part of it's usual content-streaming technology. Just download it in the background while you play.

Problem with automatic download with ArialUnicodeMS is that Afga Monotype has an exclusive license with Microsoft for it's distrobution. (ie; no-one else is allowed to distrobute it).

It's probably going to be a better deal in the long run, if LL goes with a 3rd-party full-unicode font, that is licensed for distrobution.

-Adam

Yes, that's the original story I heard about that project, I think. Another article about the "Mission to Arabic" project is here:

Mission to Arabic: It's Not Your Father’s Language Lab. (2004). University of California.
Retrieved May 9, 2004 from the World Wide Web: http://www.isi.edu/stories/78.html

I think we can do better in SL.


Thanks, I'll check that out.

I've used Rosetta Stone, as well as a wide range of other language-learning software, mostly for learning Mandarin Chinese. As you may know, Chinese is a tonal language, and doesn't derive from Indo-European languages. Rosetta Stone was really developed for European languages, and the lessons don't work well for Chinese (there's a whole unit on plurals, which Chinese doesn't have), and the pronunciation guide fails miserably on tones and seems to be more focused on meter than correct consonants and vowels. Besides which, it's pretty boring after a while -- the motivational problem you mentioned.

Also, the lack of specific feedback has been shown to be a problem with all of the language programs, or more correctly, the addition of constructive, specific feedback has been shown to substantially improve learning. Actually, I've done a fair amount of research in this area. If you're really interested, here's a link to a paper I wrote: http://gaeacoop.org/dalton/publications/Tech-Chinese-Proposal.pdf

I hadn't seen Second Life at this point, so I proposed implementing a language learning system using a MUD. SL would be much better, I think, if we could get scriptable NPCs and an appropriate API to handle processing the speech audio. (Comparing the effectiveness of the two would make a *really* interesting research study....)

neko

On a Mac, using Simplified Chinese input methods, I can nihao, but I cannot xiexie. Xiexie comes up as little blocks. OS X is theoretically Unicode-ready. It would be nice to be able to use Chinese input methods, though... trying to enter Chinese from the Unicode hex input is next to impossible.

On the plus side, I can now ask my Swedish friend hur är det?

Nice job, guys!

I agree. This is something I'm very interested in working on. The idea you discuss in your paper are definitely relevant to an SL language instruction implementation. IM me in world some time, or maybe we could move this discussion to another thread since it's getting a bit off the topic of 1.6. Also, I plan to put on some discussion events at my Psychology group headquarters about language and learning soon - hopefully in the next week. I suspect there may be quite a lot of interest in these topics among the SL userbase, given how many countries it has people from.

As for 1.6, I very much hope that Unicode support in such as in scripts, notecards and other text fields will be fully implemented, or at least somewhat more completely implemented by by the time of the 1.6 release.

Alexis and I have started a Linguistics and Languages group. We'll be looking at language learning tools as well as other fun ways to use language in SL, especially now with the new Unicode support. Feel free to sign up and join in the fun!

neko

I accidentally forgot to set the membership of the Linguistics and Languages group to "open." I've fixed that now, in case anyone is interested....

neko