Gudasoft

Impossible is nothing

Thinks that I want to read

http://scpd.stanford.edu/knuth/

Full Text search with MySQL

Goodbye MySQL

I was optimistic that I could make http://www.cenite.com, a price monitoring website to use the fulltext search of mysql. Unfortuntly I have found so many drawbacks that I have to leave this idea. The main source for information for me was:

http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html

http://devzone.zend.com/node/view/id/1304#Heading14

At first the speed was wonderfull. I was searching in 300k> records apx. 350mb. But then I have to surrender. I cant configure mysql to work as I want. I know that If I spend two days to become expert on C/C++ with Unicode I will success but this is not the case. I want working solution.

The resons that make me not to use mysql for searching:

There is no way to change the default operator by default it is OR. You must parse the user query and rewrite it.

I want automaticaly truncation on all my terms.

There is no way to tell MySQL what are characters, and what are not…..sorry, there are two ways:

1. Touching the sources,

2. Configuring in xmls

No documentation on both. Maybe there is …somewhere.

If you use the default configuration then you will wonder how you get or not get the required results.

Here is a summary of the comands that I use to tweak my mysql server:

SHOW VARIABLES LIKE ‘ft%’
SET @global.ft_min_word_len=2;
SET @local.ft_min_word_len=2;

But it is better to have those options in my.conf
[mysqld]
ft_min_word_len=3
ft_stopword_file=”C:\MySQL\stop.txt”

[myisamchk]
ft_min_word_len=3
ft_stopword_file=”C:\MySQL\stop.txt”

To check what characterset is using your mysql:

SHOW VARIABLES LIKE ‘character_sets_dir’;

/usr/share/mysql/charsets | E:MySQL Server 5.1sharecharsets

SHOW VARIABLES LIKE ‘characte%’
After that you need to rebuild your index with one of those commands
slow: REPAIR TABLE products QUICK;
slow: myisamchk –recover –ft_min_word_len=3 tbl_name.MYI
fastest: DROP INDEX …; CREATE INDEX….;

Alternatives

http://www.sphinxsearch.com/

http://endeca.com/

http://lucene.apache.org/solr/

Everyday SQL statements

Tools

Status

SHOW status where Variable_name like ‘Th%’ or Variable_name like ‘%Connec%’ ;
SHOW [GLOBAL | SESSION] STATUS [LIKE 'pattern' | WHERE expr]

Check/Repair tables

mysqlcheck -u root -p***** –auto-repair –check –optimize –all-databases

Profiling

watch -n 0.5 ‘mysql -u root -ppass -e “SHOW FULL PROCESSLIST” | grep Query’

http://opendba.blogspot.com/2008/03/mysql-finally-ability-to-traceprofile.html

mysql> set profiling=1;
mysql> select count(*) from mysql.user;
mysql> show profile;

Dump

pg_dump -U test arachnid_archiv_test –inserts -h chaos.spider.bg –encoding=utf8 -f pgsql.sql

mysqldump -c -h $MYSQL_HOST -u $MYSQL_USER -p$MYSQL_PASS -r $tfile --add-drop-table $DB
mysqldump -c -h localhost -u system3 system3_production -psomepassword -r system3_production.sql  --add-drop-table $DB

Dump for full backup with flushing of the log files

mysqldump -h $MYSQL_HOST -u $MYSQL_USER -p$MYSQL_PASS
–single-transaction –all-databases –delete-master-logs –flush-logs –master-data=2
> backup_sunday_1_PM.sql

Encoding problems

http://www.hostbulgaria.com/tutorials/mysql-charset-encoding.aspx

SHOW VARIABLES LIKE ‘character_set_%’;
curl -i http://system3.spider.bg

Creating a database

create database re_production DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

Creating a user

GRANT ALL PRIVILEGES ON arachnid_production.* TO ‘payak’@'%’ IDENTIFIED BY ‘payakpassword’ WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON system3_production.* TO ‘payak’@'%’ IDENTIFIED BY ‘payakpassword’ WITH GRANT OPTION;

mysqladmin -u [user] -h localhost -p password ‘[new_password]‘

SQL for a table

SHOW CREATE TABLE tblname;

mysql tunel to another machine

ssh -N -f -l root -L 0.0.0.0:3307:91.196.240.132:3306 s1
open port 3307 on the local machine to 91.196.240.132:3306 and login into s1 with root

Replace text

UPDATE script_histories SET cod_script = replace(cod_script,”observer.ArchiveObserver(siteId)”,”observer.ArchiveObserver(siteId, script_id, owned_source_id)”);

Copy from one table to another

DELETE FROM system3_production.articles;
INSERT INTO system3_production.articles SELECT * FROM arachnid_from_screen.articles;

Sessions for Rails

select count(*) from sessions where updated_at < DATE_SUB(now(), INTERVAL 3 DAY);

Binnary loging

http://dev.mysql.com/doc/refman/5.0/en/recovery-from-backups.html

Check this attachment here: mysql-presentation on replication etc.

Configuration

max_allowed_packet = 50M
wait_timeout=720
max_connections=1000
connect_timeout=20

query_cache_limit=8M #~~~ removed, 1M def. max pozwl. razmer za cache-hirane na edna zajawka
query_cache_size=128M #~~~ 32M, 0 def.
query_cache_type=1

Restoring the maintian Debian User

GRANT ALL PRIVILEGES ON *.* TO 'debian-sys-maint'@'localhost' IDENTIFIED BY '<password>' WITH GRANT OPTION;
Replace <password> with your debian-sys-maint password.

will_paginate with ajax

It is very easy to handle ajax pagination as described on this post here.

I want to give one full example.

Here is how my form looks:

<% form_remote_tag :url => {
                      :action => 'handle_urls',
                      :script_id => @script,
                      :page => params[:page],
                    },
  :html => {:id => 'f'},
  :update => "filtered_articles_pane" do %>

Here is how my will_paginate helper looks:

<%= will_paginate @filtered_urls,
  :renderer => 'RemoteLinkRenderer',
  :remote => { :with => "'script_id=#{@script.id}&approved_flag='+$(\'approved_flag\')",
  :update => 'filtered_articles_pane'} %>

I have used the improved version of the will_paginate:

# Use it like so…
# will_paginate :collection, :remote => {:with => 'value', :update => 'some_div'} 

class RemoteLinkRenderer < WillPaginate::LinkRenderer
  def initialize(collection, options, template)
    @remote = options.delete(:remote)
    super
  end

  def page_link_or_span(page, span_class = 'current', text = nil)
    text ||= page.to_s
    if page and page != current_page
      @template.link_to_remote(text, {:url => url_options(page), :method => :get}.merge(@remote))
    else
      @template.content_tag :span, text, :class => span_class
    end
  end
end

Update for will_paginate 2.3.x

# app/helpers/remote_link_renderer.rb

class RemoteLinkRenderer < WillPaginate::LinkRenderer
  def prepare(collection, options, template)
    @remote = options.delete(:remote) || {}
    super
  end

protected
  def page_link(page, text, attributes = {})
    @template.link_to_remote(text, {:url => url_for(page), :method => :get}.merge(@remote))
  end
end

Rails 2.0 scaffolding

The most valuable usage for scaffolding is when you use it in the administration part of the site. The administration is usually separated in separate module/folder.

The new rails scaffolding is not good because:

I find Akita article for scaffolding. It is is great that he has covered some real cases. It is a lot of reading and I must read/copy/paste every time I need to make a scaffold. Here is step by step on making scaffold for administration.

Traps:

The order of the routes in routes.rb does matter! In order to make the REST work you must have this block at the bottom of your routes.rb like this:

map.namespace :admin do |admin|
  admin.resources :bad_articles
end
# Install the default route as the lowest priority.
map.connect ':controller/:action/:id.:format'
map.connect ':controller/:action/:id'


Remove simply_helpful plugin if you get the error:

undefined method `new_record?' for #Array:0x532a3e8

Debuging tools:

rake routes

Tips:

How to add custom actions to the REST

map.namespace :admin do |admin|
  admin..esources :bad_articles, :member => {:toggle => :get}
  admin.resources :orders, :member => { :resend => :post }
  admin.resources :users,  :collection => { :filter => :any }
end

Then in the controller:

def toggle
  @bad_article = BadArticle.find(params[:id])
  @bad_article.fixed = ! @bad_article.fixed
  @bad_article.save
  redirect_to  admin_bad_articles_url
end

And in the view:

<%= toggle_admin_bad_article_url(filtered_url) %>

More reading here

Resources:

Rails plugins, gems, search engines, applications

Plugins & gems lists

Associations

http://www.workingwithrails.com/railsplugin/4801-has-many-polymorphs

Versioning of AR

http://github.com/fatjam/acts_as_revisable/tree/master

http://opensoul.org/2006/7/21/acts_as_audited

Scafolding

http://streamlinedframework.org/pages/about

Ajax pagination with JQuery

http://ozmm.org/posts/ajax_will_paginate_jq_style.html

Find_by_param is a nice and easy way to handle permalinks and dealing with searching for to_param values

http://github.com/bumi/find_by_param/tree/master

Asset Plugin – better than rails 2.0 integrated

http://synthesis.sbecker.net/pages/asset_packager

Image Magic

http://vantulder.net/rails/magick/

OpenID

http://wiki.rubyonrails.org/rails/pages/OpenidLoginGenerator

http://github.com/technoweenie/restful-authentication/tree/master

http://github.com/mrflip/

Model graph visualize

Article on franzens.org

Simple Captcha

http://expressica.com/simple_captcha/

Memcached stuff

http://townx.org/rails_and_memcached

Good rails 2.1 overview

memcached -vv -l 127.0.0.1 -p 11211 -m 256 -P /tmp/memcached.pid -u mongrel

memcached-tool 127.0.0.1

monitor the connections with the hidden option in the memcached-tool

echo $(($(netstat -nt | grep 11211 | grep -v WAIT | wc -l)/2)); ./memcached-tool 127.0.0.1:11211 dump > memdump; cat memdump

memcached-tool

alternative memcached client

PDF Output

http://ruby-pdf.rubyforge.org/pdf-writer/

http://rubyforge.org/projects/railspdfplugin/

Somone example report

Need to be checked: http://code.rubyreports.org/

Reporting

ruport & acts_as_reportable
some java reporting into rails

Sending Email

Inline email attachments plugin: http://flow.handle.it/past/2007/11/5/inline_attachment_now_official_rocks/

Wiki integration

http://github.com/queso/signal-wiki/tree/master

Repositories

http://github.com/mrflip/

Tutorials

Rails2.0 Video http://www.vimeo.com/425800

http://media.rubyonrails.org/video/rails_take2_with_sound.mov

http://akitaonrails.com/2007/12/12/rolling-with-rails-2-0-the-first-full-tutorial

http://akitaonrails.com/2007/12/12/rolling-with-rails-2-0-the-first-full-tutorial-part-2

Performance and Loging

http://austinentrepreneur.wordpress.com/2008/06/21/announcing-clientperf-simple-client-side-rails-performance/

web statistics

http://www.railstat.com/wiki/FAQ

http://haveamint.com/about/feature_highlights – not free

production log analyzers

webstat like: http://watson.rubyforge.org/

speed: http://github.com/wvanbergen/request-log-analyzer/wikis

speed: http://ckhsponge.wordpress.com/2006/10/11/ruby-on-rails-log-analyzer-rawk/

pl-analyse: http://seattlerb.rubyforge.org/production_log_analyzer http://seattlerb.rubyforge.org/production_log_analyzer

Open source rails projects

Server setup

capistrano

mod_rails

nginx

Online games

http://www.newgrounds.com/portal/view/422855

Dual Monitors with NVidia

aptitude install nvidia-glx-new
aptitude install nvidia-settings

Some options that you can put in the Screen section

/etc/X11/xorg.conf
Option “NoLogo” “True”
Option “RenderAccel” “True”
# This turns on NVidias TwinView
Option “TwinView”
# Option “MetaModes” “1280×1024″ “1152×864″ “1024×768″
# LeftOf, RightOf, Below, Above, or Clone.
# Option “TwinViewOrientation” “LeftOf”

Fun links

flip text: http://www.revfad.com/flip.html

cats: http://i011.radikal.ru/0805/da/61ea5e6e5c0c.gif

drawing graffity - http://graffiti.playdo.com/

Тест за география http://www.travelpod.com/traveler-iq

Game

Once in the dark DOS ages I have one game with which you can make wonderfull animations. In the “Windows” era I miss this game a lot….until I found this nice replacement.

http://phun.cs.umu.se/wiki/Download