gemoji 3.0.0.rc2

These images are finally no longer identical.

.gitignore

@@ -1,2 +1,9 @@
+/bin/*
+!/bin/gemoji
+.bundle
 .ruby-version
-db/NamesList.txt
+Gemfile.lock
+db/emoji-test.txt
+db/ucd.nounihan.grouped.xml
+images/unicode/*.png
+vendor/

.travis.yml

@@ -1,5 +1,9 @@
+sudo: false
+script: script/test
 rvm:
   - 1.9.3
-  - 2.1.2
+  - '2.1'
+  - '2.2'
+  - 2.3.0
 notifications:
   email: false

CONTRIBUTING.md

@@ -0,0 +1,52 @@
+This project adheres to the [Open Code of Conduct][code-of-conduct]. By participating, you are expected to uphold this code.
+[code-of-conduct]: http://todogroup.org/opencodeofconduct/#gemoji/opensource@github.com
+
+Our emoji set is based off Apple's emoji character palette, plus some custom
+emoji such as :octocat: :shipit: :metal:.
+
+Some useful tools in development are:
+
+```
+script/bootstrap
+```
+
+Sets up the development environment. The prerequisites are:
+
+* Ruby 1.9+
+* Bundler
+
+```
+rake db:generate
+```
+
+On OS X, this will rebuild the `db/Category-Emoji.json` file from the system
+one, pulling in any new emoji that Apple may have added in the meantime.
+
+```
+script/test
+```
+
+Runs the test suite, including the integrity test where we assert that we have
+covered each of Apple's emoji.
+
+```
+script/regenerate
+```
+
+Rebuilds the `db/emoji.json` file which is our main list of emoji: their
+canonical representations, descriptions, aliases, and tags. This requires OS X
+because Safari is used in the process to verify which character render as emoji
+and which render as ordinary Unicode glyphs from the current font.
+
+```
+script/console
+```
+
+Opens `irb` console with gemoji library preloded for experimentation.
+
+```
+script/release
+```
+
+For maintainers only: after the gemspec has been edited, this commits the
+change, tags a release, and pushes it to both GitHub and RubyGems.org.

Gemfile

@@ -1,6 +1,6 @@
 source "https://rubygems.org"
 
-gem "rake"
-gem "minitest"
+gem "rake", "~> 10.3.2"
+gem "minitest", "~> 5.3.5"
 
 gemspec

Gemfile.lock

@@ -1,18 +0,0 @@
-PATH
-  remote: .
-  specs:
-    gemoji (1.5.0)
-
-GEM
-  remote: https://rubygems.org/
-  specs:
-    minitest (5.3.5)
-    rake (10.3.2)
-
-PLATFORMS
-  ruby
-
-DEPENDENCIES
-  gemoji!
-  minitest
-  rake

LICENSE

@@ -10,9 +10,6 @@ Copyright (c) 2013 id Software. All rights reserved.
 trollface
 Copyright (c) 2013 whynne@deviantart. All rights reserved.
 
-All other images
-Copyright (c) 2013 Apple Inc. All rights reserved.
-
 Source code:
 
 Copyright (c) 2013 GitHub, Inc.

MAINTAINING.md

@@ -1,27 +0,0 @@
-# Maintainers
-
-## Releasing a new gem
-
-If you are just adding new emoji or making a small fix, only increment the patch level "1.0.x". If you need to rename a ton of emojis or making any other radical (but still mostly backwards compatible changes), bump the minor version "1.x.x".
-
-### Make a release commit
-
-To prepare the release commit, edit the [gemoji.gemspec](https://github.com/github/gemoji/blob/master/gemoji.gemspec) `version` value. Then make a single commit with the description as "Gemoji 1.x.x". Finally, tag the commit with `v1.x.x`.
-
-Example commit https://github.com/github/gemoji/commit/v1.0.0
-
-```
-$ git ci -m "Gemoji 1.0.0"
-$ git tag v1.0.0
-$ git push
-$ git push --tags
-```
-
-### Publish the gem
-
-Build and push the new gem with
-
-```
-$ gem build gemoji.gemspec
-$ gem push gemoji-1.0.0.gem
-```

README.md

@@ -1,7 +1,8 @@
 gemoji
 ======
 
-Emoji images and names. See the LICENSE for copyright information.
+This library contains character information about native emoji, as well as image
+files for a few custom emoji.
 
 
 Installation
@@ -13,36 +14,19 @@ Add `gemoji` to your Gemfile.
 gem 'gemoji'
 ```
 
-**Sync images**
+### Extract images
 
-Images can be copied to your public directory with `rake emoji` in your app. This is the recommended approach since the images will be available at a consistent location. This works best with cached formatted user content generated by tools like [html-pipeline](https://github.com/jch/html-pipeline).
+To obtain image files as fallbacks for browsers and OS's that don't support
+emoji, run the `gemoji extract` command **on macOS Sierra or later**:
 
-``` ruby
-# Rakefile
-load 'tasks/emoji.rake'
+``` sh
+bundle exec gemoji extract public/images/emoji
 ```
 
-```
-$ rake emoji
-```
+This will extract images into filenames such as:
 
-**Assets Precompiling**
-
-If you must, you can manually add all the images to your asset load path.
-
-``` ruby
-# config/application.rb
-config.assets.paths << Emoji.images_path
-```
-
-Then have them compiled to public on deploy.
-
-``` ruby
-# config/application.rb
-config.assets.precompile << "emoji/**/*.png"
-```
-
-**WARNING** Since there are a ton of images, just adding the path may slow down other lookups if you aren't using it. Compiling all the emojis on deploy will add overhead to your deploy if even the images haven't changed. Theres just so many more superfluous files to iterate over. Also, the urls will be fingerprinted which may not be ideal for referencing from cached content.
+* `public/images/emoji/octocat.png`
+* `public/images/emoji/unicode/1f9c0.png` (the `:cheese:` emoji)
 
 
 Example Rails Helper
@@ -56,8 +40,8 @@ See the [Emoji cheat sheet](http://www.emoji-cheat-sheet.com) for more examples.
 module EmojiHelper
   def emojify(content)
     h(content).to_str.gsub(/:([\w+-]+):/) do |match|
-      if emoji = Emoji.find_by_alias($1) { nil }
-        %(<img alt="#$1" src="#{asset_path("emoji/#{emoji.image_filename}")}" style="vertical-align:middle" width="20" height="20" />)
+      if emoji = Emoji.find_by_alias($1)
+        %(<img alt="#$1" src="#{image_path("emoji/#{emoji.image_filename}")}" style="vertical-align:middle" width="20" height="20" />)
       else
         match
       end
@@ -107,6 +91,14 @@ emoji.image_filename #=> "music.png"
 As you create new emoji, you must ensure that you also create and put the images
 they reference by their `image_filename` to your assets directory.
 
+You can customize `image_filename` with:
+
+```ruby
+emoji = Emoji.create("music") do |char|
+  char.image_filename = "subdirectory/my_emoji.gif"
+end
+```
+
 For existing emojis, you can edit the list of aliases or add new tags in an edit block:
 
 ```ruby

Rakefile

@@ -9,7 +9,11 @@ end
 
 namespace :db do
   desc %(Generate Emoji data files needed for development)
-  task :generate => ['db/Category-Emoji.json', 'db/NamesList.txt']
+  task :generate => [
+    'db/Category-Emoji.json',
+    'db/ucd.nounihan.grouped.xml',
+    'db/emoji-test.txt',
+  ]
 
   desc %(Dump a list of supported Emoji with Unicode descriptions and aliases)
   task :dump => :generate do
@@ -17,13 +21,25 @@ namespace :db do
   end
 end
 
-emoji_plist = '/System/Library/Input Methods/CharacterPalette.app/Contents/Resources/Category-Emoji.plist'
-nameslist_url = 'http://www.unicode.org/Public/6.3.0/ucd/NamesList.txt'
-
 task 'db/Category-Emoji.json' do |t|
-  system "plutil -convert json -r '#{emoji_plist}' -o '#{t.name}'"
+  system 'plutil', '-convert', 'json', '-r',
+    '/System/Library/Input Methods/CharacterPalette.app/Contents/Resources/Category-Emoji.plist',
+    '-o', t.name
 end
 
-file 'db/NamesList.txt' do |t|
-  system "curl -fsSL '#{nameslist_url}' -o '#{t.name}'"
+file 'db/ucd.nounihan.grouped.xml' do
+  Dir.chdir('db') do
+    system 'curl', '-fsSLO', 'http://www.unicode.org/Public/9.0.0/ucdxml/ucd.nounihan.grouped.zip'
+    system 'unzip', '-q', 'ucd.nounihan.grouped.zip'
+    rm 'ucd.nounihan.grouped.zip'
+  end
+end
+
+file 'db/emoji-test.txt' do |t|
+  system 'curl', '-fsSL', 'http://unicode.org/Public/emoji/4.0/emoji-test.txt', '-o', t.name
+end
+
+directory 'images/unicode' do
+  require 'emoji/extractor'
+  Emoji::Extractor.new(64, Emoji.images_path).extract!
 end

bin/gemoji

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+require 'emoji/cli'
+
+exit_code = Emoji::CLI.dispatch(ARGV)
+exit exit_code

db/Category-Emoji.json

@@ -4,40 +4,64 @@
       "CVDataTitle" : "EmojiCategory-People",
       "CVCategoryImage" : "Emoji-HumanImage",
       "CVCategoryData" : {
-        "CVSkipNullGlyphs" : true,
-        "Data" : "😄,😃,😀,😊,☺️,😉,😍,😘,😚,😗,😙,😜,😝,😛,😳,😁,😔,😌,😒,😞,😣,😢,😂,😭,😪,😥,😰,😅,😓,😩,😫,😨,😱,😠,😡,😤,😖,😆,😋,😷,😎,😴,😵,😲,😟,😦,😧,😈,👿,😮,😬,😐,😕,😯,😶,😇,😏,😑,👲,👳,👮,👷,💂,👶,👦,👧,👨,👩,👴,👵,👱,👼,👸,😺,😸,😻,😽,😼,🙀,😿,😹,😾,👹,👺,🙈,🙉,🙊,💀,👽,💩,🔥,✨,🌟,💫,💥,💢,💦,💧,💤,💨,👂,👀,👃,👅,👄,👍,👎,👌,👊,✊,✌️,👋,✋,👐,👆,👇,👉,👈,🙌,🙏,☝️,👏,💪,🚶,🏃,💃,👫,👪,👬,👭,💏,💑,👯,🙆,🙅,💁,🙋,💆,💇,💅,👰,🙎,🙍,🙇,🎩,👑,👒,👟,👞,👡,👠,👢,👕,👔,👚,👗,🎽,👖,👘,👙,💼,👜,👝,👛,👓,🎀,🌂,💄,💛,💙,💜,💚,❤️,💔,💗,💓,💕,💖,💞,💘,💌,💋,💍,💎,👤,👥,💬,👣,💭"
+        "CVSkipNullGlyphs" : false,
+        "Data" : "😀,😃,😄,😁,😆,😅,😂,🤣,☺️,😊,😇,🙂,🙃,😉,😌,😍,😘,😗,😙,😚,😋,😜,😝,😛,🤑,🤗,🤓,😎,🤡,🤠,😏,😒,😞,😔,😟,😕,🙁,☹️,😣,😖,😫,😩,😤,😠,😡,😶,😐,😑,😯,😦,😧,😮,😲,😵,😳,😱,😨,😰,😢,😥,🤤,😭,😓,😪,😴,🙄,🤔,🤥,😬,🤐,🤢,🤧,😷,🤒,🤕,😈,👿,👹,👺,💩,👻,💀,☠️,👽,👾,🤖,🎃,😺,😸,😹,😻,😼,😽,🙀,😿,😾,👐,🙌,👏,🙏,🤝,👍,👎,👊,✊,🤛,🤜,🤞,✌️,🤘,👌,👈,👉,👆,👇,☝️,✋,🤚,🖐,🖖,👋,🤙,💪,🖕,✍️,🤳,💅,💍,💄,💋,👄,👅,👂,👃,👣,👁,👀,🗣,👤,👥,👶,👦,👧,👨,👩,👱‍♀️,👱,👴,👵,👲,👳‍♀️,👳,👮‍♀️,👮,👷‍♀️,👷,💂‍♀️,💂,🕵️‍♀️,🕵️,👩‍⚕️,👨‍⚕️,👩‍🌾,👨‍🌾,👩‍🍳,👨‍🍳,👩‍🎓,👨‍🎓,👩‍🎤,👨‍🎤,👩‍🏫,👨‍🏫,👩‍🏭,👨‍🏭,👩‍💻,👨‍💻,👩‍💼,👨‍💼,👩‍🔧,👨‍🔧,👩‍🔬,👨‍🔬,👩‍🎨,👨‍🎨,👩‍🚒,👨‍🚒,👩‍✈️,👨‍✈️,👩‍🚀,👨‍🚀,👩‍⚖️,👨‍⚖️,🤶,🎅,👸,🤴,👰,🤵,👼,🤰,🙇‍♀️,🙇,💁,💁‍♂️,🙅,🙅‍♂️,🙆,🙆‍♂️,🙋,🙋‍♂️,🤦‍♀️,🤦‍♂️,🤷‍♀️,🤷‍♂️,🙎,🙎‍♂️,🙍,🙍‍♂️,💇,💇‍♂️,💆,💆‍♂️,🕴,💃,🕺,👯,👯‍♂️,🚶‍♀️,🚶,🏃‍♀️,🏃,👫,👭,👬,💑,👩‍❤️‍👩,👨‍❤️‍👨,💏,👩‍❤️‍💋‍👩,👨‍❤️‍💋‍👨,👪,👨‍👩‍👧,👨‍👩‍👧‍👦,👨‍👩‍👦‍👦,👨‍👩‍👧‍👧,👩‍👩‍👦,👩‍👩‍👧,👩‍👩‍👧‍👦,👩‍👩‍👦‍👦,👩‍👩‍👧‍👧,👨‍👨‍👦,👨‍👨‍👧,👨‍👨‍👧‍👦,👨‍👨‍👦‍👦,👨‍👨‍👧‍👧,👩‍👦,👩‍👧,👩‍👧‍👦,👩‍👦‍👦,👩‍👧‍👧,👨‍👦,👨‍👧,👨‍👧‍👦,👨‍👦‍👦,👨‍👧‍👧,👚,👕,👖,👔,👗,👙,👘,👠,👡,👢,👞,👟,👒,🎩,🎓,👑,⛑,🎒,👝,👛,👜,💼,👓,🕶,🌂,☂️"
       }
     },
     {
       "CVDataTitle" : "EmojiCategory-Nature",
       "CVCategoryImage" : "Emoji-NatureImage",
       "CVCategoryData" : {
-        "CVSkipNullGlyphs" : true,
-        "Data" : "🐶,🐺,🐱,🐭,🐹,🐰,🐸,🐯,🐨,🐻,🐷,🐽,🐮,🐗,🐵,🐒,🐴,🐑,🐘,🐼,🐧,🐦,🐤,🐥,🐣,🐔,🐍,🐢,🐛,🐝,🐜,🐞,🐌,🐙,🐚,🐠,🐟,🐬,🐳,🐋,🐄,🐏,🐀,🐃,🐅,🐇,🐉,🐎,🐐,🐓,🐕,🐖,🐁,🐂,🐲,🐡,🐊,🐫,🐪,🐆,🐈,🐩,🐾,💐,🌸,🌷,🍀,🌹,🌻,🌺,🍁,🍃,🍂,🌿,🌾,🍄,🌵,🌴,🌲,🌳,🌰,🌱,🌼,🌐,🌞,🌝,🌚,🌑,🌒,🌓,🌔,🌕,🌖,🌗,🌘,🌜,🌛,🌙,🌍,🌎,🌏,🌋,🌌,🌠,⭐️,☀️,⛅️,☁️,⚡️,☔️,❄️,⛄️,🌀,🌁,🌈,🌊"
+        "CVSkipNullGlyphs" : false,
+        "Data" : "🐶,🐱,🐭,🐹,🐰,🦊,🐻,🐼,🐨,🐯,🦁,🐮,🐷,🐽,🐸,🐵,🙈,🙉,🙊,🐒,🐔,🐧,🐦,🐤,🐣,🐥,🦆,🦅,🦉,🦇,🐺,🐗,🐴,🦄,🐝,🐛,🦋,🐌,🐚,🐞,🐜,🕷,🕸,🐢,🐍,🦎,🦂,🦀,🦑,🐙,🦐,🐠,🐟,🐡,🐬,🦈,🐳,🐋,🐊,🐆,🐅,🐃,🐂,🐄,🦌,🐪,🐫,🐘,🦏,🦍,🐎,🐖,🐐,🐏,🐑,🐕,🐩,🐈,🐓,🦃,🕊,🐇,🐁,🐀,🐿,🐾,🐉,🐲,🌵,🎄,🌲,🌳,🌴,🌱,🌿,☘️,🍀,🎍,🎋,🍃,🍂,🍁,🍄,🌾,💐,🌷,🌹,🥀,🌻,🌼,🌸,🌺,🌎,🌍,🌏,🌕,🌖,🌗,🌘,🌑,🌒,🌓,🌔,🌚,🌝,🌞,🌛,🌜,🌙,💫,⭐️,🌟,✨,⚡️,🔥,💥,☄️,☀️,🌤,⛅️,🌥,🌦,🌈,☁️,🌧,⛈,🌩,🌨,☃️,⛄️,❄️,🌬,💨,🌪,🌫,🌊,💧,💦,☔️"
       }
     },
     {
-      "CVDataTitle" : "EmojiCategory-Objects",
-      "CVCategoryImage" : "Emoji-ObjectsImage",
+      "CVDataTitle" : "EmojiCategory-Foods",
+      "CVCategoryImage" : "Emoji-FoodsImage",
       "CVCategoryData" : {
-        "CVSkipNullGlyphs" : true,
-        "Data" : "🎍,💝,🎎,🎒,🎓,🎏,🎆,🎇,🎐,🎑,🎃,👻,🎅,🎄,🎁,🎋,🎉,🎊,🎈,🎌,🔮,🎥,📷,📹,📼,💿,📀,💽,💾,💻,📱,☎️,📞,📟,📠,📡,📺,📻,🔊,🔉,🔈,🔇,🔔,🔕,📢,📣,⏳,⌛️,⏰,⌚️,🔓,🔒,🔏,🔐,🔑,🔎,💡,🔦,🔆,🔅,🔌,🔋,🔍,🛁,🛀,🚿,🚽,🔧,🔩,🔨,🚪,🚬,💣,🔫,🔪,💊,💉,💰,💴,💵,💷,💶,💳,💸,📲,📧,📥,📤,✉️,📩,📨,📯,📫,📪,📬,📭,📮,📦,📝,📄,📃,📑,📊,📈,📉,📜,📋,📅,📆,📇,📁,📂,✂️,📌,📎,✒️,✏️,📏,📐,📕,📗,📘,📙,📓,📔,📒,📚,📖,🔖,📛,🔬,🔭,📰,🎨,🎬,🎤,🎧,🎼,🎵,🎶,🎹,🎻,🎺,🎷,🎸,👾,🎮,🃏,🎴,🀄️,🎲,🎯,🏈,🏀,⚽️,⚾️,🎾,🎱,🏉,🎳,⛳️,🚵,🚴,🏁,🏇,🏆,🎿,🏂,🏊,🏄,🎣,☕️,🍵,🍶,🍼,🍺,🍻,🍸,🍹,🍷,🍴,🍕,🍔,🍟,🍗,🍖,🍝,🍛,🍤,🍱,🍣,🍥,🍙,🍘,🍚,🍜,🍲,🍢,🍡,🍳,🍞,🍩,🍮,🍦,🍨,🍧,🎂,🍰,🍪,🍫,🍬,🍭,🍯,🍎,🍏,🍊,🍋,🍒,🍇,🍉,🍓,🍑,🍈,🍌,🍐,🍍,🍠,🍆,🍅,🌽"
+        "CVSkipNullGlyphs" : false,
+        "Data" : "🍏,🍎,🍐,🍊,🍋,🍌,🍉,🍇,🍓,🍈,🍒,🍑,🍍,🥝,🥑,🍅,🍆,🥒,🥕,🌽,🌶,🥔,🍠,🌰,🥜,🍯,🥐,🍞,🥖,🧀,🥚,🍳,🥓,🥞,🍤,🍗,🍖,🍕,🌭,🍔,🍟,🥙,🌮,🌯,🥗,🥘,🍝,🍜,🍲,🍥,🍣,🍱,🍛,🍚,🍙,🍘,🍢,🍡,🍧,🍨,🍦,🍰,🎂,🍮,🍭,🍬,🍫,🍿,🍩,🍪,🥛,🍼,☕️,🍵,🍶,🍺,🍻,🥂,🍷,🥃,🍸,🍹,🍾,🥄,🍴,🍽"
+      }
+    },
+    {
+      "CVDataTitle" : "EmojiCategory-Activity",
+      "CVCategoryImage" : "Emoji-ActivityImage",
+      "CVCategoryData" : {
+        "CVSkipNullGlyphs" : false,
+        "Data" : "⚽️,🏀,🏈,⚾️,🎾,🏐,🏉,🎱,🏓,🏸,🥅,🏒,🏑,🏏,⛳️,🏹,🎣,🥊,🥋,⛸,🎿,⛷,🏂,🏋️‍♀️,🏋️,🤺,🤼‍♀️,🤼‍♂️,🤸‍♀️,🤸‍♂️,⛹️‍♀️,⛹️,🤾‍♀️,🤾‍♂️,🏌️‍♀️,🏌️,🏄‍♀️,🏄,🏊‍♀️,🏊,🤽‍♀️,🤽‍♂️,🚣‍♀️,🚣,🏇,🚴‍♀️,🚴,🚵‍♀️,🚵,🎽,🏅,🎖,🥇,🥈,🥉,🏆,🏵,🎗,🎫,🎟,🎪,🤹‍♀️,🤹‍♂️,🎭,🎨,🎬,🎤,🎧,🎼,🎹,🥁,🎷,🎺,🎸,🎻,🎲,🎯,🎳,🎮,🎰"
       }
     },
     {
       "CVDataTitle" : "EmojiCategory-Places",
       "CVCategoryImage" : "Emoji-PlacesImage",
       "CVCategoryData" : {
-        "CVSkipNullGlyphs" : true,
-        "Data" : "🏠,🏡,🏫,🏢,🏣,🏥,🏦,🏪,🏩,🏨,💒,⛪️,🏬,🏤,🌇,🌆,🏯,🏰,⛺️,🏭,🗼,🗾,🗻,🌄,🌅,🌃,🗽,🌉,🎠,🎡,⛲️,🎢,🚢,⛵️,🚤,🚣,⚓️,🚀,✈️,💺,🚁,🚂,🚊,🚉,🚞,🚆,🚄,🚅,🚈,🚇,🚝,🚋,🚃,🚎,🚌,🚍,🚙,🚘,🚗,🚕,🚖,🚛,🚚,🚨,🚓,🚔,🚒,🚑,🚐,🚲,🚡,🚟,🚠,🚜,💈,🚏,🎫,🚦,🚥,⚠️,🚧,🔰,⛽️,🏮,🎰,♨️,🗿,🎪,🎭,📍,🚩,🇯🇵,🇰🇷,🇩🇪,🇨🇳,🇺🇸,🇫🇷,🇪🇸,🇮🇹,🇷🇺,🇬🇧"
+        "CVSkipNullGlyphs" : false,
+        "Data" : "🚗,🚕,🚙,🚌,🚎,🏎,🚓,🚑,🚒,🚐,🚚,🚛,🚜,🛴,🚲,🛵,🏍,🚨,🚔,🚍,🚘,🚖,🚡,🚠,🚟,🚃,🚋,🚞,🚝,🚄,🚅,🚈,🚂,🚆,🚇,🚊,🚉,🚁,🛩,✈️,🛫,🛬,🚀,🛰,💺,🛶,⛵️,🛥,🚤,🛳,⛴,🚢,⚓️,🚧,⛽️,🚏,🚦,🚥,🗺,🗿,🗽,⛲️,🗼,🏰,🏯,🏟,🎡,🎢,🎠,⛱,🏖,🏝,⛰,🏔,🗻,🌋,🏜,🏕,⛺️,🛤,🛣,🏗,🏭,🏠,🏡,🏘,🏚,🏢,🏬,🏣,🏤,🏥,🏦,🏨,🏪,🏫,🏩,💒,🏛,⛪️,🕌,🕍,🕋,⛩,🗾,🎑,🏞,🌅,🌄,🌠,🎇,🎆,🌇,🌆,🏙,🌃,🌌,🌉,🌁"
+      }
+    },
+    {
+      "CVDataTitle" : "EmojiCategory-Objects",
+      "CVCategoryImage" : "Emoji-ObjectsImage",
+      "CVCategoryData" : {
+        "CVSkipNullGlyphs" : false,
+        "Data" : "⌚️,📱,📲,💻,⌨️,🖥,🖨,🖱,🖲,🕹,🗜,💽,💾,💿,📀,📼,📷,📸,📹,🎥,📽,🎞,📞,☎️,📟,📠,📺,📻,🎙,🎚,🎛,⏱,⏲,⏰,🕰,⌛️,⏳,📡,🔋,🔌,💡,🔦,🕯,🗑,🛢,💸,💵,💴,💶,💷,💰,💳,💎,⚖️,🔧,🔨,⚒,🛠,⛏,🔩,⚙️,⛓,🔫,💣,🔪,🗡,⚔️,🛡,🚬,⚰️,⚱️,🏺,🔮,📿,💈,⚗️,🔭,🔬,🕳,💊,💉,🌡,🚽,🚰,🚿,🛁,🛀,🛎,🔑,🗝,🚪,🛋,🛏,🛌,🖼,🛍,🛒,🎁,🎈,🎏,🎀,🎊,🎉,🎎,🏮,🎐,✉️,📩,📨,📧,💌,📥,📤,📦,🏷,📪,📫,📬,📭,📮,📯,📜,📃,📄,📑,📊,📈,📉,🗒,🗓,📆,📅,📇,🗃,🗳,🗄,📋,📁,📂,🗂,🗞,📰,📓,📔,📒,📕,📗,📘,📙,📚,📖,🔖,🔗,📎,🖇,📐,📏,📌,📍,✂️,🖊,🖋,✒️,🖌,🖍,📝,✏️,🔍,🔎,🔏,🔐,🔒,🔓"
       }
     },
     {
       "CVDataTitle" : "EmojiCategory-Symbols",
       "CVCategoryImage" : "Emoji-SymbolImage",
       "CVCategoryData" : {
-        "CVSkipNullGlyphs" : true,
-        "Data" : "1️⃣,2️⃣,3️⃣,4️⃣,5️⃣,6️⃣,7️⃣,8️⃣,9️⃣,0️⃣,🔟,🔢,#️⃣,🔣,⬆️,⬇️,⬅️,➡️,🔠,🔡,🔤,↗️,↖️,↘️,↙️,↔️,↕️,🔄,◀️,▶️,🔼,🔽,↩️,↪️,ℹ️,⏪,⏩,⏫,⏬,⤵️,⤴️,🆗,🔀,🔁,🔂,🆕,🆙,🆒,🆓,🆖,📶,🎦,🈁,🈯️,🈳,🈵,🈴,🈲,🉐,🈹,🈺,🈶,🈚️,🚻,🚹,🚺,🚼,🚾,🚰,🚮,🅿️,♿️,🚭,🈷,🈸,🈂,Ⓜ️,🛂,🛄,🛅,🛃,🉑,㊙️,㊗️,🆑,🆘,🆔,🚫,🔞,📵,🚯,🚱,🚳,🚷,🚸,⛔️,✳️,❇️,❎,✅,✴️,💟,🆚,📳,📴,🅰,🅱,🆎,🅾,💠,➿,♻️,♈️,♉️,♊️,♋️,♌️,♍️,♎️,♏️,♐️,♑️,♒️,♓️,⛎,🔯,🏧,💹,💲,💱,©,®,™,❌,‼️,⁉️,❗️,❓,❕,❔,⭕️,🔝,🔚,🔙,🔛,🔜,🔃,🕛,🕧,🕐,🕜,🕑,🕝,🕒,🕞,🕓,🕟,🕔,🕠,🕕,🕖,🕗,🕘,🕙,🕚,🕡,🕢,🕣,🕤,🕥,🕦,✖️,➕,➖,➗,♠️,♥️,♣️,♦️,💮,💯,✔️,☑️,🔘,🔗,➰,〰,〽️,🔱,◼️,◻️,◾️,◽️,▪️,▫️,🔺,🔲,🔳,⚫️,⚪️,🔴,🔵,🔻,⬜️,⬛️,🔶,🔷,🔸,🔹"
+        "CVSkipNullGlyphs" : false,
+        "Data" : "❤️,💛,💚,💙,💜,🖤,💔,❣️,💕,💞,💓,💗,💖,💘,💝,💟,☮️,✝️,☪️,🕉,☸️,✡️,🔯,🕎,☯️,☦️,🛐,⛎,♈️,♉️,♊️,♋️,♌️,♍️,♎️,♏️,♐️,♑️,♒️,♓️,🆔,⚛️,🉑,☢️,☣️,📴,📳,🈶,🈚️,🈸,🈺,🈷️,✴️,🆚,💮,🉐,㊙️,㊗️,🈴,🈵,🈹,🈲,🅰️,🅱️,🆎,🆑,🅾️,🆘,❌,⭕️,🛑,⛔️,📛,🚫,💯,💢,♨️,🚷,🚯,🚳,🚱,🔞,📵,🚭,❗️,❕,❓,❔,‼️,⁉️,🔅,🔆,〽️,⚠️,🚸,🔱,⚜️,🔰,♻️,✅,🈯️,💹,❇️,✳️,❎,🌐,💠,Ⓜ️,🌀,💤,🏧,🚾,♿️,🅿️,🈳,🈂️,🛂,🛃,🛄,🛅,🚹,🚺,🚼,🚻,🚮,🎦,📶,🈁,🔣,ℹ️,🔤,🔡,🔠,🆖,🆗,🆙,🆒,🆕,🆓,0️⃣,1️⃣,2️⃣,3️⃣,4️⃣,5️⃣,6️⃣,7️⃣,8️⃣,9️⃣,🔟,🔢,#️⃣,*️⃣,▶️,⏸,⏯,⏹,⏺,⏭,⏮,⏩,⏪,⏫,⏬,◀️,🔼,🔽,➡️,⬅️,⬆️,⬇️,↗️,↘️,↙️,↖️,↕️,↔️,↪️,↩️,⤴️,⤵️,🔀,🔁,🔂,🔄,🔃,🎵,🎶,➕,➖,➗,✖️,💲,💱,™️,©️,®️,〰️,➰,➿,🔚,🔙,🔛,🔝,🔜,✔️,☑️,🔘,⚪️,⚫️,🔴,🔵,🔺,🔻,🔸,🔹,🔶,🔷,🔳,🔲,▪️,▫️,◾️,◽️,◼️,◻️,⬛️,⬜️,🔈,🔇,🔉,🔊,🔔,🔕,📣,📢,👁‍🗨,💬,💭,🗯,♠️,♣️,♥️,♦️,🃏,🎴,🀄️,🕐,🕑,🕒,🕓,🕔,🕕,🕖,🕗,🕘,🕙,🕚,🕛,🕜,🕝,🕞,🕟,🕠,🕡,🕢,🕣,🕤,🕥,🕦,🕧"
+      }
+    },
+    {
+      "CVDataTitle" : "EmojiCategory-Flags",
+      "CVCategoryImage" : "Emoji-FlagsImage",
+      "CVCategoryData" : {
+        "CVSkipNullGlyphs" : false,
+        "Data" : "🏳️,🏴,🏁,🚩,🏳️‍🌈,🇦🇫,🇦🇽,🇦🇱,🇩🇿,🇦🇸,🇦🇩,🇦🇴,🇦🇮,🇦🇶,🇦🇬,🇦🇷,🇦🇲,🇦🇼,🇦🇺,🇦🇹,🇦🇿,🇧🇸,🇧🇭,🇧🇩,🇧🇧,🇧🇾,🇧🇪,🇧🇿,🇧🇯,🇧🇲,🇧🇹,🇧🇴,🇧🇶,🇧🇦,🇧🇼,🇧🇷,🇮🇴,🇻🇬,🇧🇳,🇧🇬,🇧🇫,🇧🇮,🇨🇻,🇰🇭,🇨🇲,🇨🇦,🇮🇨,🇰🇾,🇨🇫,🇹🇩,🇨🇱,🇨🇳,🇨🇽,🇨🇨,🇨🇴,🇰🇲,🇨🇬,🇨🇩,🇨🇰,🇨🇷,🇨🇮,🇭🇷,🇨🇺,🇨🇼,🇨🇾,🇨🇿,🇩🇰,🇩🇯,🇩🇲,🇩🇴,🇪🇨,🇪🇬,🇸🇻,🇬🇶,🇪🇷,🇪🇪,🇪🇹,🇪🇺,🇫🇰,🇫🇴,🇫🇯,🇫🇮,🇫🇷,🇬🇫,🇵🇫,🇹🇫,🇬🇦,🇬🇲,🇬🇪,🇩🇪,🇬🇭,🇬🇮,🇬🇷,🇬🇱,🇬🇩,🇬🇵,🇬🇺,🇬🇹,🇬🇬,🇬🇳,🇬🇼,🇬🇾,🇭🇹,🇭🇳,🇭🇰,🇭🇺,🇮🇸,🇮🇳,🇮🇩,🇮🇷,🇮🇶,🇮🇪,🇮🇲,🇮🇱,🇮🇹,🇯🇲,🇯🇵,🎌,🇯🇪,🇯🇴,🇰🇿,🇰🇪,🇰🇮,🇽🇰,🇰🇼,🇰🇬,🇱🇦,🇱🇻,🇱🇧,🇱🇸,🇱🇷,🇱🇾,🇱🇮,🇱🇹,🇱🇺,🇲🇴,🇲🇰,🇲🇬,🇲🇼,🇲🇾,🇲🇻,🇲🇱,🇲🇹,🇲🇭,🇲🇶,🇲🇷,🇲🇺,🇾🇹,🇲🇽,🇫🇲,🇲🇩,🇲🇨,🇲🇳,🇲🇪,🇲🇸,🇲🇦,🇲🇿,🇲🇲,🇳🇦,🇳🇷,🇳🇵,🇳🇱,🇳🇨,🇳🇿,🇳🇮,🇳🇪,🇳🇬,🇳🇺,🇳🇫,🇲🇵,🇰🇵,🇳🇴,🇴🇲,🇵🇰,🇵🇼,🇵🇸,🇵🇦,🇵🇬,🇵🇾,🇵🇪,🇵🇭,🇵🇳,🇵🇱,🇵🇹,🇵🇷,🇶🇦,🇷🇪,🇷🇴,🇷🇺,🇷🇼,🇧🇱,🇸🇭,🇰🇳,🇱🇨,🇵🇲,🇻🇨,🇼🇸,🇸🇲,🇸🇹,🇸🇦,🇸🇳,🇷🇸,🇸🇨,🇸🇱,🇸🇬,🇸🇽,🇸🇰,🇸🇮,🇸🇧,🇸🇴,🇿🇦,🇬🇸,🇰🇷,🇸🇸,🇪🇸,🇱🇰,🇸🇩,🇸🇷,🇸🇿,🇸🇪,🇨🇭,🇸🇾,🇹🇼,🇹🇯,🇹🇿,🇹🇭,🇹🇱,🇹🇬,🇹🇰,🇹🇴,🇹🇹,🇹🇳,🇹🇷,🇹🇲,🇹🇨,🇹🇻,🇺🇬,🇺🇦,🇦🇪,🇬🇧,🇺🇸,🇻🇮,🇺🇾,🇺🇿,🇻🇺,🇻🇦,🇻🇪,🇻🇳,🇼🇫,🇪🇭,🇾🇪,🇿🇲,🇿🇼"
       }
     }
   ],

db/aliases.applescript

@@ -0,0 +1,13 @@
+set jsCode to "document.getElementById('output').value"
+set json to missing value
+
+tell application "Safari"
+  repeat
+    set json to (do JavaScript jsCode in current tab of window 1)
+    if (json is not missing value) then exit repeat
+    delay 0.5
+  end repeat
+  close current tab of window 1
+end tell
+
+return json

db/aliases.html

@@ -0,0 +1,81 @@
+<!DOCTYPE html>
+<title>Emoji alias detection</title>
+<style>
+  textarea {
+    font-family: monospace;
+  }
+</style>
+
+<p>Save the following as <tt>emoji.json</tt>:</p>
+
+<textarea id="output" rows="50" cols="80"></textarea>
+
+<script>
+  const VARIATION_SELECTOR_15 = String.fromCharCode(0xfe0e);
+  const VARIATION_SELECTOR_16 = String.fromCharCode(0xfe0f);
+  const EMOJI_SIZE = 32
+
+  function detectAliases(db) {
+    for (var i = 0; i < db.length; ++i) {
+      var emoji = db[i];
+      var raw = emoji.emoji;
+      if (!raw) {
+        continue;
+      }
+
+      if (raw.indexOf(VARIATION_SELECTOR_16) > -1) {
+        var candidates = [raw.replace(VARIATION_SELECTOR_16, ""), raw];
+      } else {
+        var candidates = [raw, raw + VARIATION_SELECTOR_16];
+      }
+
+      var aliases = candidates.filter(isColorEmoji);
+      emoji.emoji = aliases[0];
+    }
+
+    dump(db);
+  }
+
+  function isColorEmoji(candidate) {
+    // Draw the emoji twice using a different color each time. If the emoji
+    // draws as the same color regardless of what color we set, it's a color
+    // emoji.
+    return color(candidate, "#f00") === color(candidate, "#0f0");
+  }
+
+  var canvas = document.createElement("canvas");
+  canvas.width = canvas.height = EMOJI_SIZE;
+
+  function color(emoji, rgb) {
+    var context = canvas.getContext("2d");
+    context.clearRect(0, 0, canvas.width, canvas.height);
+    context.fillStyle = rgb;
+    context.textBaseline = "top";
+    context.font = EMOJI_SIZE+"px Arial";
+    context.fillText(emoji, 0, 0);
+    var data = context.getImageData(0, 0, EMOJI_SIZE, EMOJI_SIZE).data;
+    for (var i = 0; i < data.length; i += 4) {
+      if (data[i] === 0 && data[i + 1] === 0 && data[i + 2] === 0) {
+        continue;
+      }
+      return data[i].toString(16)
+        + data[i + 1].toString(16)
+        + data[i + 2].toString(16);
+    }
+    return "no colored pixel found";
+  }
+
+  function dump(db) {
+    var json = JSON.stringify(db, null, "  ")
+      .replace(/^( +)(.+)\[\](,?)$/mg, "$1$2[\n$1]$3")
+      .replace(/,\n( *)  /g, "\n$1, ");
+    document.getElementById("output").value = json + "\n";
+  }
+
+  var xhr = new XMLHttpRequest;
+  xhr.onload = function() {
+    detectAliases(JSON.parse(this.responseText));
+  };
+  xhr.open("GET", "emoji.json", false);
+  xhr.send(null);
+</script>

db/dump.rb

@@ -1,28 +1,57 @@
 require 'emoji'
 require 'json'
-
-names_list = File.expand_path('../NamesList.txt', __FILE__)
+require 'rexml/document'
 
 class UnicodeCharacter
-  attr_reader :code, :description, :aliases
+  attr_reader :code, :description, :version, :aliases
 
-  @index = {}
-  class << self
-    attr_reader :index
+  class CharListener
+    CHAR_TAG = "char".freeze
 
-    def fetch(code, *args, &block)
-      code = code.to_s(16).rjust(4, '0') if code.is_a?(Integer)
-      index.fetch(code, *args, &block)
+    def self.parse(io, &block)
+      REXML::Document.parse_stream(io, self.new(&block))
     end
+
+    def initialize(&block)
+      @callback = block
+    end
+
+    def tag_start(name, attributes)
+      if CHAR_TAG == name
+        @callback.call(
+          attributes.fetch("cp") { return },
+          attributes.fetch("na") { return },
+          attributes.fetch("age", nil),
+        )
+      end
+    end
+
+    def method_missing(*) end
   end
 
-  def initialize(code, description)
+  def self.index
+    return @index if defined? @index
+    @index = {}
+    File.open(File.expand_path('../ucd.nounihan.grouped.xml', __FILE__)) do |source|
+      CharListener.parse(source) do |char, desc, age|
+        uc = UnicodeCharacter.new(char, desc, age)
+        @index[uc.code] = uc
+      end
+    end
+    @index
+  end
+
+  def self.fetch(code)
+    code = code.to_s(16).rjust(4, '0') if code.is_a?(Integer)
+    self.index.fetch(code)
+  end
+
+  def initialize(code, description, version)
     @code = code.downcase
     @description = description.downcase
+    @version = version
     @aliases = []
     @references = []
-
-    self.class.index[@code] = self
   end
 
   def add_alias(string)
@@ -34,44 +63,74 @@ class UnicodeCharacter
   end
 end
 
-char = nil
-
-File.foreach(names_list) do |line|
-  case line
-  when /^[A-F0-9]{4,5}\t/
-    code, desc = line.chomp.split("\t", 2)
-    codepoint = code.hex
-    char = UnicodeCharacter.new(code, desc)
-  when /^\t= /
-    char.add_alias($')
-  when /^\tx .+ - ([A-F0-9]{4,5})\)$/
-    char.add_reference($1)
+unless $stdin.tty?
+  codepoints = STDIN.read.chomp.codepoints.map { |code|
+    UnicodeCharacter.fetch(code)
+  }
+  codepoints.each do |char|
+    printf "%5s: %s", char.code.upcase, char.description
+    printf " (%s)", char.version if char.version
+    puts
   end
+  exit
 end
 
 trap(:PIPE) { abort }
 
-items = []
-variation = Emoji::VARIATION_SELECTOR_16
+normalize = -> (raw) {
+  raw.sub(Emoji::VARIATION_SELECTOR_16, '')
+}
 
-for emoji in Emoji.all
-  unicodes = emoji.unicode_aliases.dup
-
-  item = {}
-
-  unless emoji.custom?
-    variation_codepoint = variation.codepoints[0]
-    chars = emoji.raw.codepoints.map { |code| UnicodeCharacter.fetch(code) unless code == variation_codepoint }.compact
-    unicodes.select { |u| u.index(variation) }.each { |u| unicodes.delete(u.sub(variation, '')) }
-    item[:emoji] = unicodes.shift
-    item[:unicodes] = unicodes if unicodes.any?
-    item[:description] = chars.map(&:description).join(' + ')
+emojidesc = {}
+File.open(File.expand_path('../emoji-test.txt', __FILE__)) do |file|
+  file.each do |line|
+    next if line =~ /^(#|$)/
+    line = line.chomp.split('# ', 2)[1]
+    emoji, description = line.split(' ', 2)
+    emojidesc[normalize.(emoji)] = description
   end
+end
 
-  item[:aliases] = emoji.aliases
-  item[:tags] = emoji.tags
+items = []
 
-  items << item
+for category, emojis in Emoji.apple_palette
+  for raw in emojis
+    emoji = Emoji.find_by_unicode(raw)
+    unicode_version = emoji ? emoji.unicode_version : ''
+    ios_version = emoji ? emoji.ios_version : ''
+
+    unless raw.include?(Emoji::ZERO_WIDTH_JOINER)
+      uchar = UnicodeCharacter.fetch(raw.codepoints[0])
+      unicode_version = uchar.version unless uchar.version.nil?
+    end
+
+    description = emojidesc.fetch(normalize.(raw))
+
+    if unicode_version == ''
+      warn "#{description} (#{raw}) doesn't have Unicode version"
+    end
+
+    if ios_version == ''
+      ios_version = '10.2'
+    end
+
+    items << {
+      emoji: raw,
+      description: description,
+      category: category,
+      aliases: emoji ? emoji.aliases : [description.gsub(/\W+/, '_').downcase],
+      tags: emoji ? emoji.tags : [],
+      unicode_version: unicode_version,
+      ios_version: ios_version,
+    }
+  end
+end
+
+for emoji in Emoji.all.select(&:custom?)
+  items << {
+    aliases: emoji.aliases,
+    tags: emoji.tags,
+  }
 end
 
 puts JSON.pretty_generate(items)

gemoji.gemspec

@@ -1,8 +1,9 @@
 Gem::Specification.new do |s|
   s.name    = "gemoji"
-  s.version = "2.0.0"
-  s.summary = "Emoji conversion and image assets"
-  s.description = "Image assets and character information for emoji."
+  s.version = "3.0.0.rc2"
+  s.summary = "Emoji library"
+  s.description = "Character information and metadata for standard and custom emoji."
+  s.executables = ["gemoji"]
 
   s.required_ruby_version = '> 1.9'
 
@@ -13,9 +14,10 @@ Gem::Specification.new do |s|
 
   s.files = Dir[
     "README.md",
-    "images/**/*.png",
+    "bin/gemoji",
+    "images/*.png",
+    "db/Category-Emoji.json",
     "db/emoji.json",
     "lib/**/*.rb",
-    "lib/tasks/*.rake"
   ]
 end