Showing posts with label indexing. Show all posts
Showing posts with label indexing. Show all posts

Sunday, March 11, 2012

Does indexing on IMAGE fields work?

I am doing some prototyping using FTS in SQL 2005 Standard and am having
serious problems getting searching to work when using IMAGE or VarBinary(max)
fields. The index sortof works when using varchar fields. The only problem
with varchar is that it doesn't seem to always auto popluate when I make a
change (see other posting "FT Index doesn't Auto Populate"). However, when
trying the same thing with Image fields loaded with RTF data, the search
always returns nothing. The crawl log shows an error when indexing is first
done; (Error '0x80004005' ), however, it states that it will attempt to
reindex. The log shows 3 more attempts which fail and then finally one that
seems to succeed showing 6 records processed and 0 failed. Aside from that,
there is no other indication of a failure in any of the steps.
Some things I've tried;
- I've upgraded to SP1
- I'm always careful to enable the index after creating it (a new feature in
SP1)
- I've tried loading the DocType field for the associated image field with
several different types (doc, rtf, txt) and tried loading all three types of
documents into the image field. None work
- I've confirmed the iFilters are loaded on this server (eventhough I know
they should be there anyway as preloaded filters)
- I've created an index on a varchar(100) field that is in this same
database and table. The indexing and searching work fine, so, we know
there's nothing odd about the FTS, database, or table
Here's one of the basic queries I use to do the search, although, I've tried
several variations (none work)
SELECT Resume
FROM tbl_ServiceProviders
WHERE CONTAINS(Resume, ' "a*" ');
Having no practical experience with this Microsoft FTS feature, the obvious
question I'm starting to consider; Is this product ready for primetime? Has
anyone gone to production with SQL 2005 FTS? I'm being asked to use this to
build a search feature for an existing "job candidate searching system". I'm
getting a little concerned.
Thanks in advance for any insights anyone can offer
Mike
Maz
Have you looked at the full-text crawl logs? Also make sure the value of the
type column corresponds to the actual file type in the varbinary/image column.
See if this helps:
http://milambda.blogspot.com/2005/12/ifilter-error-bydesign-or-bybug.html
BTW, when searching for "a*" are you searching or words beginning with the
letter a? If you do, try again without the double quotes.
ML
http://milambda.blogspot.com/
|||Thanks ML for directing me to the article. Unfortunately, I've considered
all those elementary suggestions already.
I'm very frustrated with this product. I haven't even started implementing
a business solution for this yet! I'm just trying to do some very basic
prototyping and can't get 1) cataloging jobs to run, 2) auto populate to work
in realtime on a consistent basis, 3) Image fields to index.
Of course you might be thinking I might be a moron. I've considered that
myself However, I can assure you I am not. I've been programming for 20+
years and working with SQL Server for 10 years. I've never had such a hard
time performing basic prototyping using the provided technical documentation.
My conclusion so far; this product is terrible. It probably works, but you
need the "real world" manual to be productive with it. Unfortunately, it
seems so few people are using this product that there doesn't seem to be a
book available to offer real practical advice. The MS documentation only
instructs you on the theoretical behavior of this product. I can only
imagine the problems we will run into in a production environment with this.
Anyone know of any books based on practical experience using SQl 2005 FTS?
If not, I'm considering looking elsewere for my full text search needs.
Mike
"ML" wrote:

> Have you looked at the full-text crawl logs? Also make sure the value of the
> type column corresponds to the actual file type in the varbinary/image column.
> See if this helps:
> http://milambda.blogspot.com/2005/12/ifilter-error-bydesign-or-bybug.html
> BTW, when searching for "a*" are you searching or words beginning with the
> letter a? If you do, try again without the double quotes.
>
> ML
> --
> http://milambda.blogspot.com/
|||Spicy wrote on Sat, 3 Feb 2007 12:55:09 -0800:

> Having no practical experience with this Microsoft FTS feature, the
> obvious question I'm starting to consider; Is this product ready for
> primetime? Has anyone gone to production with SQL 2005 FTS? I'm being
> asked to use this to build a search feature for an existing "job candidate
> searching system". I'm getting a little concerned.
This bit I can answer - I've been using SQL 2005 FTS on public websites
since Feb 2006, and prior to that SQL 7 FTS for a few years (we skipped 2000
on our web sites, but use 2000 FTS internally). If you check http://www.compman.co.uk
, http://www.bookfellas.co.uk , or http://www.sprintbooks.co.uk , all
searches for keywords are done using FTS on SQL 2005. This is running on a
database with almost 700,000 products, and with FTIs on various columns for
each product.
Dan
|||That's somewhat conforting to hear. I suppose I'll keep trying to search the
internet for instances of this happening. I'm following the instructions
exactly. It just won't work for me. I've even tried changing the doc type
to include the dot because I saw it used both ways in some examples (e.g.
"rtf" and ".rtf") Nothing seems to work.
Maz
"Daniel Crichton" wrote:

> Spicy wrote on Sat, 3 Feb 2007 12:55:09 -0800:
>
> This bit I can answer - I've been using SQL 2005 FTS on public websites
> since Feb 2006, and prior to that SQL 7 FTS for a few years (we skipped 2000
> on our web sites, but use 2000 FTS internally). If you check http://www.compman.co.uk
> , http://www.bookfellas.co.uk , or http://www.sprintbooks.co.uk , all
> searches for keywords are done using FTS on SQL 2005. This is running on a
> database with almost 700,000 products, and with FTIs on various columns for
> each product.
> Dan
>
>
|||So, there are no errors in the crawl log?
ML
http://milambda.blogspot.com/
|||Take a look at my first post on this thread. There is an error that states
it will retry to index. It retries (and fails) several times, then it
finally succeeds and says that 6 rows were indexed and 0 failed. That was
the only thing that seemed exceptional. However, since it finally stated
that the index succeeded, I assumed that was not the problem. FYI: There
arre 100 records in the test database and 6 rows with data in the image
field. The reported numbers seem to match up with what I expected to see
Maz
"ML" wrote:

> So, there are no errors in the crawl log?
>
> ML
> --
> http://milambda.blogspot.com/
|||We index over 2 terabytes. It works well for us. The 4005 error is a generic
access denied, it could be caused by several factors. Can you try to extract
your data as text and index it that way? I would also remove a from your
noise word list and rebuild your catalog to remove the possibility that that
is causing your problems.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Spicy Mikey" <Maz@.newsgroups.nospam> wrote in message
news:10663BF1-D3E4-4D58-8D7E-B1C9B325B5BB@.microsoft.com...
>I am doing some prototyping using FTS in SQL 2005 Standard and am having
> serious problems getting searching to work when using IMAGE or
> VarBinary(max)
> fields. The index sortof works when using varchar fields. The only
> problem
> with varchar is that it doesn't seem to always auto popluate when I make a
> change (see other posting "FT Index doesn't Auto Populate"). However,
> when
> trying the same thing with Image fields loaded with RTF data, the search
> always returns nothing. The crawl log shows an error when indexing is
> first
> done; (Error '0x80004005' ), however, it states that it will attempt to
> reindex. The log shows 3 more attempts which fail and then finally one
> that
> seems to succeed showing 6 records processed and 0 failed. Aside from
> that,
> there is no other indication of a failure in any of the steps.
> Some things I've tried;
> - I've upgraded to SP1
> - I'm always careful to enable the index after creating it (a new feature
> in
> SP1)
> - I've tried loading the DocType field for the associated image field with
> several different types (doc, rtf, txt) and tried loading all three types
> of
> documents into the image field. None work
> - I've confirmed the iFilters are loaded on this server (eventhough I know
> they should be there anyway as preloaded filters)
> - I've created an index on a varchar(100) field that is in this same
> database and table. The indexing and searching work fine, so, we know
> there's nothing odd about the FTS, database, or table
> Here's one of the basic queries I use to do the search, although, I've
> tried
> several variations (none work)
> SELECT Resume
> FROM tbl_ServiceProviders
> WHERE CONTAINS(Resume, ' "a*" ');
>
> Having no practical experience with this Microsoft FTS feature, the
> obvious
> question I'm starting to consider; Is this product ready for primetime?
> Has
> anyone gone to production with SQL 2005 FTS? I'm being asked to use this
> to
> build a search feature for an existing "job candidate searching system".
> I'm
> getting a little concerned.
> Thanks in advance for any insights anyone can offer
> Mike
>
>
> --
> Maz
|||Spicy wrote on Mon, 5 Feb 2007 06:01:02 -0800:

> That's interesting about the 4005 being an access problem. I wonder if
> all these problems are related to the same thing. In my other thread I
> talked about not being able to create the catalog job. It tells me it
> can't connect to the server remotely and suggested I be sure to have
> "remote access" turned on. I'm sure that's not the problem. I can access
> the database from my application using oledb and a standard connection
> string. I'm trying to creat the catalog from SMSS and am logged on as
> "sa".
I have no problems with SMSS or EM (for 2000/7), I am logged onto an account
on my local machine that exists as an admin on the SQL Server machine (it's
on a DMZ, not in our ADS, hence having to setup an account on the SQL
machine for me to admin it). It sounds as though you might have a problem
with the NTLM side of things - remember, the 'sa' login via SMSS is only for
SQL Server itself - communication with the Search service that is FTS is via
the NT subsystem itself, so you'll require your local login credentials to
match an admin level (or one that's able to communicate with the Search
service) credential on the SQL Server machine.
Dan
|||It is reported as an generic access problem, but it could be something
entirely different.
If you want to send me sample docs off line, I'll try to repro your problem.
We might get to the bottom of it faster this way.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Spicy Mikey" <Maz@.newsgroups.nospam> wrote in message
news:CA379695-8D0C-46E7-8C65-F0F0AE6C5153@.microsoft.com...[vbcol=seagreen]
> That's interesting about the 4005 being an access problem. I wonder if
> all
> these problems are related to the same thing. In my other thread I talked
> about not being able to create the catalog job. It tells me it can't
> connect
> to the server remotely and suggested I be sure to have "remote access"
> turned
> on. I'm sure that's not the problem. I can access the database from my
> application using oledb and a standard connection string. I'm trying to
> creat the catalog from SMSS and am logged on as "sa".
> By the way, I tried several different search strings, not just "a*".
> That's
> not the problem. Rest assured, I've tried anything obvious and logical.
> Since FTS works great for you, I am assuming there is some "quirk" in the
> product that is burning me. Something that you can't know about from
> reading the documentation.
> --
> Maz
>
> "Hilary Cotter" wrote:

Friday, March 9, 2012

Does full text indexing works on unicode field?

Hi all,
Does full text indexing supports searching on a unicode field, for
example, a field that stores chinese characters?
Thank you in advance.Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"alvinyyt" <alvinyyt@.gmail.com> wrote in message
news:1144720305.221875.221920@.e56g2000cwe.googlegroups.com...
> Hi all,
> Does full text indexing supports searching on a unicode field, for
> example, a field that stores chinese characters?
> Thank you in advance.
>|||Yes, make sure you select the appropriate language word breaker.
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"alvinyyt" <alvinyyt@.gmail.com> wrote in message
news:1144720305.221875.221920@.e56g2000cwe.googlegroups.com...
> Hi all,
> Does full text indexing supports searching on a unicode field, for
> example, a field that stores chinese characters?
> Thank you in advance.
>

Does FTS in SQL 7 have known issues with not indexing records?

I've been testing using a full text index on a few columns in one of my
databases, and I'm having trouble with the index appearing to miss records.
I have stopped all updates on my database, and done a full population of the
SearchTitle field in my STK table. I have then waited until the full text
update has completed, and there are no errors in the event log. I then tried
the following queries:
SELECT STK.ID, STK.SearchTitle FROM STK WHERE
CONTAINS(STK.SearchTitle,'"being" and "jordan"')
Result is zero records. I also tried
SELECT STK.ID, STK.SearchTitle FROM STK WHERE
CONTAINS(STK.SearchTitle,'"being jordan"')
Again, zero records. I then tried
SELECT STK.ID, STK.SearchTitle FROM STK WHERE STK.SearchTitle LIKE '% being
%' and STK.SearchTitle LIKE '% jordan %')
and get 1 result, which is what I expect.
The SearchTitle field contains a stripped down version of book titles in my
table, all fields have a space followed by the words in the table followed
by an ending space (this is so that the current searches I do via the last
example work on whole words without ever finding partial matches). In the
above case the SearchTitle field contains just ' being jordan ' (without the
quotes).
SearchTitle is a varchar(255) field, and there are just under 365572 rows in
the table. The FT index shows 339632 items with 380112 unique words. I have
emptied the noise word files because they were causing problems with
searches, so I know it's not a noise word issue. This indicates that FTS has
skipped around 26000 records. I am currently running another full population
to see if the problem is a temporary one, but I was wondering if there are
known issues with FT indexing that I might be experiencing.
Dan
Being could be a noise word for the noise word list. Do you get the same
number of hits if you search on Jordan as you get if you search on Like '%
Jordan %'?
"Daniel Crichton" <msnews@.worldofspack.co.uk> wrote in message
news:urZxCI5rEHA.1204@.TK2MSFTNGP12.phx.gbl...
> I've been testing using a full text index on a few columns in one of my
> databases, and I'm having trouble with the index appearing to miss
> records.
> I have stopped all updates on my database, and done a full population of
> the
> SearchTitle field in my STK table. I have then waited until the full text
> update has completed, and there are no errors in the event log. I then
> tried
> the following queries:
> SELECT STK.ID, STK.SearchTitle FROM STK WHERE
> CONTAINS(STK.SearchTitle,'"being" and "jordan"')
> Result is zero records. I also tried
> SELECT STK.ID, STK.SearchTitle FROM STK WHERE
> CONTAINS(STK.SearchTitle,'"being jordan"')
> Again, zero records. I then tried
> SELECT STK.ID, STK.SearchTitle FROM STK WHERE STK.SearchTitle LIKE '%
> being
> %' and STK.SearchTitle LIKE '% jordan %')
> and get 1 result, which is what I expect.
> The SearchTitle field contains a stripped down version of book titles in
> my
> table, all fields have a space followed by the words in the table followed
> by an ending space (this is so that the current searches I do via the last
> example work on whole words without ever finding partial matches). In the
> above case the SearchTitle field contains just ' being jordan ' (without
> the
> quotes).
> SearchTitle is a varchar(255) field, and there are just under 365572 rows
> in
> the table. The FT index shows 339632 items with 380112 unique words. I
> have
> emptied the noise word files because they were causing problems with
> searches, so I know it's not a noise word issue. This indicates that FTS
> has
> skipped around 26000 records. I am currently running another full
> population
> to see if the problem is a temporary one, but I was wondering if there are
> known issues with FT indexing that I might be experiencing.
> Dan
>
|||"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:u71T0k6rEHA.3076@.TK2MSFTNGP10.phx.gbl...
> Being could be a noise word for the noise word list. Do you get the same
> number of hits if you search on Jordan as you get if you search on Like '%
> Jordan %'?
No, instead of getting the expected 58 titles, I get 54. I have cleared the
noise word list before generating the index - SQL was throwing out errors if
1 noise word was passed into the search even if there were other non-noise
words, so I decided that rather than parsing out the noise words and getting
in-exact matches for what customers enter in their searches I'd just index
everything.
Dan
|||After running a full population again it appears to have now indexed
everything. I'll be doing some more preliminary testing before putting this
live though, last thing I want is for customers not to be able to find items
in our database (the example I gave of Being Jordan was the top selling book
a few weeks ago, not having that listed in search results would have been
disastrous for us.
Dan
|||"Daniel Crichton" <msnews@.worldofspack.co.uk> wrote in message
news:%23jpG03CsEHA.3748@.TK2MSFTNGP09.phx.gbl...
> After running a full population again it appears to have now indexed
> everything. I'll be doing some more preliminary testing before putting
this
> live though, last thing I want is for customers not to be able to find
items
> in our database (the example I gave of Being Jordan was the top selling
book
> a few weeks ago, not having that listed in search results would have been
> disastrous for us.
Looking at the event logs I've noticed that despite the item count being
correct, in the information event at completion of an incremental update
there is:
"Detected end of incremental crawl for project <SQLServer SQL0002300005>.
Successfully processed 365614 documents, 0K. Failed to filter 13 documents.
Modified 282 documents."
Followed by a warning event with ID 3051 containing:
"Detected 13 URLs that could not be reached or were denied access in project
<SQLServer SQL0002300005>."
I assume that for some reason 13 items couldn't be accessed when running the
incremental search. I'm running another one now to see if I get the same
messages, as it's only taking around 20 mins to run the incremental compared
to 4 hours running the full population.
I've just looked back at the full population I ran yesterday, and have
noticed that it also logged a warning event (I have updated some records
since this was built, hence the difference in the item counts). Here's the
information one first:
"Detected end of crawl for project <SQLServer SQL0002300005>. Successfully
processed 365652 documents, 0K. Failed to filter 0 documents."
Followed by a warning event:
"Detected 365452 URLs that could not be reached or were denied access in
project <SQLServer SQL0002300005>."
Whereas the previous pair of errors made sense in that the information
message indicates that 13 records couldn't be indexed, and the warning seems
to confirm this, the pair for the full population are confusing in that they
don't match. Does this indicate a potential problem in the indexing system?
Or is the logging of mismatched item counts in the event log a normal
occurrence?
Dan
|||make sure your noise word list has a single space in it, otherwise it will
be using the noise word list found in %windir%\system32.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
"Daniel Crichton" <msnews@.worldofspack.co.uk> wrote in message
news:urhcK2CsEHA.324@.TK2MSFTNGP11.phx.gbl...[vbcol=seagreen]
> "Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
> news:u71T0k6rEHA.3076@.TK2MSFTNGP10.phx.gbl...
'%
> No, instead of getting the expected 58 titles, I get 54. I have cleared
the
> noise word list before generating the index - SQL was throwing out errors
if
> 1 noise word was passed into the search even if there were other non-noise
> words, so I decided that rather than parsing out the noise words and
getting
> in-exact matches for what customers enter in their searches I'd just index
> everything.
> Dan
>
|||I strongly suggest you move to SQL 2000 for performance and scalability
reasons.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
"Daniel Crichton" <msnews@.worldofspack.co.uk> wrote in message
news:%23jpG03CsEHA.3748@.TK2MSFTNGP09.phx.gbl...
> After running a full population again it appears to have now indexed
> everything. I'll be doing some more preliminary testing before putting
this
> live though, last thing I want is for customers not to be able to find
items
> in our database (the example I gave of Being Jordan was the top selling
book
> a few weeks ago, not having that listed in search results would have been
> disastrous for us.
> Dan
>
|||You can get these errors for a variety of reasons.
You get 0 rows could not be indexed typically for the below reasons.
1) the account SQL Server runs under is not registered with MSSearch. You
can get this when you change the SQL Server service account through control
panel instead of via Enterprise Manager. This will cause the entire table
not to be indexed.
2) verify that the login BUILTIN\Administrator exists in the login folder.
If it does not exist add it in.
You will get xxx rows could not be indexed typically for the below reasons
1) one or more rows were deleted since the last population
2) a row was locked
3) a row could contain a very large amount of data which could not be
extracted in the time MSSearch waits to extract each row.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
"Daniel Crichton" <msnews@.worldofspack.co.uk> wrote in message
news:u4xryIDsEHA.1816@.TK2MSFTNGP15.phx.gbl...[vbcol=seagreen]
> "Daniel Crichton" <msnews@.worldofspack.co.uk> wrote in message
> news:%23jpG03CsEHA.3748@.TK2MSFTNGP09.phx.gbl...
> this
> items
> book
been
> Looking at the event logs I've noticed that despite the item count being
> correct, in the information event at completion of an incremental update
> there is:
> "Detected end of incremental crawl for project <SQLServer SQL0002300005>.
> Successfully processed 365614 documents, 0K. Failed to filter 13
documents.
> Modified 282 documents."
> Followed by a warning event with ID 3051 containing:
> "Detected 13 URLs that could not be reached or were denied access in
project
> <SQLServer SQL0002300005>."
> I assume that for some reason 13 items couldn't be accessed when running
the
> incremental search. I'm running another one now to see if I get the same
> messages, as it's only taking around 20 mins to run the incremental
compared
> to 4 hours running the full population.
>
> I've just looked back at the full population I ran yesterday, and have
> noticed that it also logged a warning event (I have updated some records
> since this was built, hence the difference in the item counts). Here's the
> information one first:
> "Detected end of crawl for project <SQLServer SQL0002300005>. Successfully
> processed 365652 documents, 0K. Failed to filter 0 documents."
> Followed by a warning event:
> "Detected 365452 URLs that could not be reached or were denied access in
> project <SQLServer SQL0002300005>."
> Whereas the previous pair of errors made sense in that the information
> message indicates that 13 records couldn't be indexed, and the warning
seems
> to confirm this, the pair for the full population are confusing in that
they
> don't match. Does this indicate a potential problem in the indexing
system?
> Or is the logging of mismatched item counts in the event log a normal
> occurrence?
> Dan
>
|||"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:eitvdMFsEHA.2340@.TK2MSFTNGP11.phx.gbl...
> I strongly suggest you move to SQL 2000 for performance and scalability
> reasons.
Unfortunately this is not an option at present due to cost - I would need
not only the SQL Server 2000 license, but also 2 SQL Server processor
licenses (dual processor server) so that the database is licensed for use on
my web sites. Last time I looked that was a hefty sum.
Dan
|||"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:eNpYhLFsEHA.2732@.TK2MSFTNGP09.phx.gbl...
> make sure your noise word list has a single space in it, otherwise it will
> be using the noise word list found in %windir%\system32.
Yes, I did leave a single space in the noise word files after reading a few
posts in here about it.
Dan

Sunday, February 19, 2012

Documentation about noise words (ignored words)

As my customers are mainly not-English-speaking, I'm used to prepare specific
versions of the file "noise.dat" for all the indexing for neutral language.
Anyway, even though I'm not including the paragraph symbol (§), I've
realized that this character is automatically considered as ignored word.
The questions are:
- Is there a way to control these ignored characters, evetually by excluding
ot including themn in the full-text catalog?
- Where can I find a detailed documentation of all these
language-independent characters that are ignored?
Thanks in advance,
Angelo
There is no real description of the noise word lists or the treatment of
special characters. In general all punctuation marks are ignored with some
exceptions.
Here is a good description of how noise words are indexed.
http://msdn.microsoft.com/library/de...nario_8k4z.asp
Hilary Cotter
Looking for a book on SQL Server replication?
http://www.nwsu.com/0974973602.html
"Angelo B" <Angelo B@.discussions.microsoft.com> wrote in message
news:87641E91-C218-4DFD-9CFF-495F5B6EF3C0@.microsoft.com...
> As my customers are mainly not-English-speaking, I'm used to prepare
specific
> versions of the file "noise.dat" for all the indexing for neutral
language.
> Anyway, even though I'm not including the paragraph symbol (), I've
> realized that this character is automatically considered as ignored word.
> The questions are:
> - Is there a way to control these ignored characters, evetually by
excluding
> ot including themn in the full-text catalog?
> - Where can I find a detailed documentation of all these
> language-independent characters that are ignored?
> Thanks in advance,
> Angelo
|||Hi Hilary
Your answer replied my question but but didn't help the issue.
The issue is that our customers are governative authorities, which make
large use of § (paragraph character) in the documents MSSearch indexes. As
this character is a prefix for law numbers, such as "the article expressed in
the § 12 of law # 2340-124", they pretend that by typing "§ 12" in the search
form they get all documents containing "§ 12", not only "12", otherwise they
get thousands of documents that have nothing to do with "paragraph 12".
I wonder why has MS chosen not to index these characters. If they exist,
they are also used in documents, and they should be searchable.
The noise.dat should give the possibility to treat these characters as
ignored word/character or not.
It's a serious issue...
Are you sure that it's not possible to include/exclude these characters? Is
there a key in the Registry, maybe?
Thanks in advance,
Angelo
"Hilary Cotter" wrote:

> There is no real description of the noise word lists or the treatment of
> special characters. In general all punctuation marks are ignored with some
> exceptions.
> Here is a good description of how noise words are indexed.
> http://msdn.microsoft.com/library/de...nario_8k4z.asp
>
> --
> Hilary Cotter
> Looking for a book on SQL Server replication?
> http://www.nwsu.com/0974973602.html
>
> "Angelo B" <Angelo B@.discussions.microsoft.com> wrote in message
> news:87641E91-C218-4DFD-9CFF-495F5B6EF3C0@.microsoft.com...
> specific
> language.
> excluding
>
>
|||I realize I was not of much help.
Someone approached me some time ago about doing something similar. What I
recommended they do is to replace all the unindexable tokens with another
token that is indexed, ie xxxx. Then when they were searching on this
unindexable character the client would replace it with the xxxx and they
would get the results they were looking for.
They had two columns storing the content, one where the unindexable
character was replaced by a searchable token (xxxx), and the other which
contains the actual content. The one where the unindexable character was
replaced was the column indexed, the other the one returned in searches.
HTH
Hilary Cotter
Looking for a book on SQL Server replication?
http://www.nwsu.com/0974973602.html
"Angelo B" <AngeloB@.discussions.microsoft.com> wrote in message
news:C6573FCA-5B09-45A8-A765-CED8891A1F81@.microsoft.com...
> Hi Hilary
> Your answer replied my question but but didn't help the issue.
> The issue is that our customers are governative authorities, which make
> large use of (paragraph character) in the documents MSSearch indexes. As
> this character is a prefix for law numbers, such as "the article expressed
in
> the 12 of law # 2340-124", they pretend that by typing " 12" in the
search
> form they get all documents containing " 12", not only "12", otherwise
they
> get thousands of documents that have nothing to do with "paragraph 12".
> I wonder why has MS chosen not to index these characters. If they exist,
> they are also used in documents, and they should be searchable.
> The noise.dat should give the possibility to treat these characters as
> ignored word/character or not.
> It's a serious issue...
> Are you sure that it's not possible to include/exclude these characters?
Is[vbcol=seagreen]
> there a key in the Registry, maybe?
> Thanks in advance,
> Angelo
>
> "Hilary Cotter" wrote:
some[vbcol=seagreen]
http://msdn.microsoft.com/library/de...nario_8k4z.asp[vbcol=seagreen]
word.[vbcol=seagreen]