Does any one have similar problem with nvarchar column storing chinese characters?

I am running into some wired problem with nvarchar(4000). When I insert the
chinese characters, the text got cut off. The following two sections shows
what happen, the first chinese section is what is ended up in the database,
the second section (separate by "--") show the complete
parameter value before IDbCommand.ExecuateQuery() is called. It is less
than 4000 bytes. It seems a chunk of text got loss somewhere.
No sure what happen here. But it seems to be a serious problem. My full
text search certainlly wouldn't work if I can't get the data into the
column. Is this a bug in 2005 CTP or something I did wrong ? Thanks
--Xin Chen
?...?? - ? - ? - ?... ? - ?Q - ?... ?
- ? - ?... ? - ? ? > ? > ? ??
?:2005?03?23?03:56?:????
?
????
???
??1937?12?13?27???,??
??,?,??
"? ?...?? - ? - ? - ?... ? - ?Q - ?... ?
? - ? - ?... ? - ? ? > ? > ? ??\n
?:2005?03?23?03:56?:????
?\n???? \n \n??
? \n ??1937?12?13?27???,?
???,?,???,??
????,??30??
?,???,??,??,??,?
???1938?3?24?,????,?
??2?31? ??,?27?,?
??,?31??,??28?,?
?31?,??29?,?2??,?
??,?2???31?,???
?4?3?,????,?
???,??,???
6?,???7?,???
?,???????
?,???,?4.6?,?7500?
??,??1????
??,??? ?\n ??
??\n ?? ????
???\n??? ??
\n?? ??\n??3-26
? ??\n?? ??\n?
?? ??\n?:? ?<?
?> \n ? ??"?"??:? ?
? ? \n -- ?? ChinaRen - ? - ? - ?
? - ? - ? - ? - ? - ?? - About SOHU -
?\nCopyright 2005 Sohu.com Inc.All rights reserved."
how are you inserting these characters? OpenRowset?
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Xin Chen" <xchen@.xtremework.com> wrote in message
news:ulgv5WANFHA.1176@.TK2MSFTNGP15.phx.gbl...
> I am running into some wired problem with nvarchar(4000). When I insert
the
> chinese characters, the text got cut off. The following two sections
shows
> what happen, the first chinese section is what is ended up in the
database,
> the second section (separate by "--") show the complete
> parameter value before IDbCommand.ExecuateQuery() is called. It is less
> than 4000 bytes. It seems a chunk of text got loss somewhere.
> No sure what happen here. But it seems to be a serious problem. My full
> text search certainlly wouldn't work if I can't get the data into the
> column. Is this a bug in 2005 CTP or something I did wrong ? Thanks
> --Xin Chen
> ?...?? - ? - ? - ?... ? - ?Q - ?... ?
> - ? - ?... ? - ? ? > ? > ? ??
> ?:2005?03?23?03:56?:????
> ?
> ????
> ???
> ??1937?12?13?27???,??
> ??,?,??
> ----
--
> --
> "? ?...?? - ? - ? - ?... ? - ?Q - ?... ?
> ? - ? - ?... ? - ? ? > ? > ? ??\n
> ?:2005?03?23?03:56?:????
> ?\n???? \n \n??
> ? \n ??1937?12?13?27???,?
> ???,?,???,??
> ????,??30??
> ?,???,??,??,??,?
> ???1938?3?24?,????,?
> ??2?31? ??,?27?,?
> ??,?31??,??28?,?
> ?31?,??29?,?2??,?
> ??,?2???31?,???
> ?4?3?,????,?
> ???,??,???
> 6?,???7?,???
> ?,???????
> ?,???,?4.6?,?7500?
> ??,??1????
> ??,??? ?\n ??
> ??\n ?? ????
> ???\n??? ??
> \n?? ??\n??3-26
> ? ??\n?? ??\n?
> ?? ??\n?:? ?<?
> ?> \n ? ??"?"??:? ?
> ? ? \n -- ?? ChinaRen - ? - ? - ?
> ? - ? - ? - ? - ? - ?? - About SOHU -
> ?\nCopyright 2005 Sohu.com Inc.All rights reserved."
>
|||no, I am using a store proc and call it via ADO.NET in my app
I store my text in the nvarchar(4000) as string, I am not using openrowset.
the stored proc use regular insert into (...) values (...)
--Xin Chen
"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:exEotkANFHA.2468@.tk2msftngp13.phx.gbl...[vbcol=seagreen]
> how are you inserting these characters? OpenRowset?
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTS
> http://www.indexserverfaq.com
> "Xin Chen" <xchen@.xtremework.com> wrote in message
> news:ulgv5WANFHA.1176@.TK2MSFTNGP15.phx.gbl...
> the
> shows
> database,
complete
> ----
> --
>

Document stored as image, problem with inflectional search

Hello,
I'm storing documents (txt, html, etc) as images in my database, and can
run regular 'contains' searches on my data just fine, but when I use
formsof(inflectional,word), it will only return results that contain the
word exactly, not any inflectional forms of it.
Is this some known issue about the way indexing is done or how SQL
processes images? Or could something be wrong on my end?
My @.@.version is:
Microsoft SQL Server 2000 - 8.00.194 (Intel X86)
Aug 6 2000 00:57:48
Copyright (c) 1988-2000 Microsoft Corporation
Enterprise Edition on Windows NT 5.0 (Build 2195: Service Pack 4)
Thanks!
- Parhaum
Parhaum,
First of all, thank you for providing the @.@.version info as this is most
important in troubleshooting SQL FTS issues such as this one! Secondly, can
you post the exact formsof(inflectional,word) query & word or phrase that
you are having problems with along with the output of the following SQL
code?
EXEC sp_help_fulltext_columns
EXEC sp_help <your_FT-enable_table_name_here>
There are several known words when used with the US English wordbreaker that
do not generate the correct or expected inflectional, see this thread
(http://www.webservertalk.com/archive.../t-965368.html) for more details.
Thanks,
John
SQL Full Text Search Blog
http://spaces.msn.com/members/jtkane/
"Parhaum Toofanian via droptable.com" <forum@.nospam.droptable.com> wrote
in message news:141e567797a94b829e022eb7c3eb7ca2@.droptable.co m...
> Hello,
> I'm storing documents (txt, html, etc) as images in my database, and can
> run regular 'contains' searches on my data just fine, but when I use
> formsof(inflectional,word), it will only return results that contain the
> word exactly, not any inflectional forms of it.
> Is this some known issue about the way indexing is done or how SQL
> processes images? Or could something be wrong on my end?
> My @.@.version is:
> Microsoft SQL Server 2000 - 8.00.194 (Intel X86)
> Aug 6 2000 00:57:48
> Copyright (c) 1988-2000 Microsoft Corporation
> Enterprise Edition on Windows NT 5.0 (Build 2195: Service Pack 4)
>
> Thanks!
> - Parhaum
|||did you use the ms.locale metatag to indicate the type of language your html
docs are stored in?
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Parhaum Toofanian via droptable.com" <forum@.nospam.droptable.com> wrote
in message news:141e567797a94b829e022eb7c3eb7ca2@.droptable.co m...
> Hello,
> I'm storing documents (txt, html, etc) as images in my database, and can
> run regular 'contains' searches on my data just fine, but when I use
> formsof(inflectional,word), it will only return results that contain the
> word exactly, not any inflectional forms of it.
> Is this some known issue about the way indexing is done or how SQL
> processes images? Or could something be wrong on my end?
> My @.@.version is:
> Microsoft SQL Server 2000 - 8.00.194 (Intel X86)
> Aug 6 2000 00:57:48
> Copyright (c) 1988-2000 Microsoft Corporation
> Enterprise Edition on Windows NT 5.0 (Build 2195: Service Pack 4)
>
> Thanks!
> - Parhaum
|||First of all, thank you both for your quick replies! Regarding the locale,
I can probably check, but it's a database in my company that I'm developing
with, so I'd need to ask the DBA about it.
Second, background on the database. The table with the image data is
J_DOCU_CONTENT, and I have uploaded many files including 3 sample .txt
files with text like "park services" and "servicing the shuttle". There's
a table called T_DOCUMENT that contains some other information, but even
running the last query without the T_DOCUMENT information returns the same
result.
Trying to do an inflectional search on "service" returns 0 results. An
inflectional search on "services" returns the park services file, and so
on, as if it were a 'contains' exact match search.
:: Result of:
:: EXEC sp_help_fulltext_columns;
TABLE_OWNER<x>
TABLE_ID<id>
TABLE_NAMEJ_DOCU_CONTENT
FULLTEXT_COLUMN_NAMEDATA
FULLTEXT_COLID4
FULLTEXT_BLOBTP_COLNAMEDOCUMENT_TYPE
FULLTEXT_BLOBTP_COLID3
FULLTEXT_LANGUAGE0
:: Result of:
:: EXEC sp_help J_DOCU_CONTENT;
NameJ_DOCU_CONTENT
Owner<x>
Typeuser table
Created_datetime2005-04-14 15:38:09.013
Column_namePRIMARY_KEY
Typeint
Computedno
Length4
Prec10
Scale0
Nullableno
TrimTrailingBlanks(n/a)
FixedLenNullInSource(n/a)
CollationNULL
Column_nameVERSION
Typeint
Computedno
Length4
Prec10
Scale0
Nullableno
TrimTrailingBlanks(n/a)
FixedLenNullInSource(n/a)
CollationNULL
Column_nameDOCUMENT_TYPE
Typevarchar
Computedno
Length250
Prec
Scale
Nullableno
TrimTrailingBlanksno
FixedLenNullInSourceno
CollationSQL_Latin1_General_CP1_CI_AS
Column_nameDATA
Typeimage
Computedno
Length7000
Prec
Scale
Nullableno
TrimTrailingBlanks(n/a)
FixedLenNullInSource(n/a)
CollationNULL
No identity column defined.
No rowguidcol column defined.
constraint_type
PRIMARY KEY (clustered)
:: Result of:
:: SELECT t1.NAME FROM J_DOCU_CONTENT t0, T_DOCUMENT t1 WHERE
::(contains(t0.DATA, ' FORMSOF(INFLECTIONAL, service) ')
::AND
::(t0.PRIMARY_KEY = t1.CONTENT_ID))
NAME
(0 row(s) affected)
Message posted via http://www.droptable.com
|||Yikes. Tried to format it from the direct SQL output, and not sure now
which ended up looking worse...
Message posted via http://www.droptable.com

Document Storage inside DB?

Has anyone got any experience or opinions about the
pros/cons of storing Office Type Documents inside an
image field in the database as opposed to storing a URL
pointer to the file in the filesystem?
Assuming there will be several GB of documents Im
thinking it would be better to keep these out of the
database.
AdamA common approach is to :-
Store the documents on a file server
Store a pointer to the Documents in the DB (to aid searching)
HTH
Ryan Waight, MCDBA, MCSE
"Adam" <atoth@.msdnspam.com> wrote in message
news:07ab01c3b4d3$2e07d430$a501280a@.phx.gbl...
> Has anyone got any experience or opinions about the
> pros/cons of storing Office Type Documents inside an
> image field in the database as opposed to storing a URL
> pointer to the file in the filesystem?
> Assuming there will be several GB of documents Im
> thinking it would be better to keep these out of the
> database.
> Adam
>|||Hi,
We are providing content management software that's using such an approach -
but we are using both DB and FileSystem as a storage for the documents. For
storing the documents inside the database, we use a "document-breaker" to
split the documents in meaningfull parts (e.g. the html documents are split
in tags and text).
The advantages we have with the documents inside the database:
- one single resource that contains the whole system
- easy backup
- depending on the document splitting, we can track and validate cross
document references
The advantages we have by also storing the documents in the file system:
- fast document access (knowing the ID, we don't need a query to the DB)
- fast search capability (using Windows Index Services)
By the way: we have productive systems using this approach with 2000+
documents.
Sven
"Adam" <atoth@.msdnspam.com> wrote in message
news:07ab01c3b4d3$2e07d430$a501280a@.phx.gbl...
> Has anyone got any experience or opinions about the
> pros/cons of storing Office Type Documents inside an
> image field in the database as opposed to storing a URL
> pointer to the file in the filesystem?
> Assuming there will be several GB of documents Im
> thinking it would be better to keep these out of the
> database.
> Adam
>|||I see two approaches that use a "file system" is this the
Windows "file system" that you are referring to and if
so... how do you insure the documents are not deleted?
>--Original Message--
>Has anyone got any experience or opinions about the
>pros/cons of storing Office Type Documents inside an
>image field in the database as opposed to storing a URL
>pointer to the file in the filesystem?
>Assuming there will be several GB of documents Im
>thinking it would be better to keep these out of the
>database.
>Adam
>.
>|||>--Original Message--
>I see two approaches that use a "file system" is this
the
>Windows "file system" that you are referring to and if
>so... how do you insure the documents are not deleted?
>
Yes - I do mean the Windows File System; in answer to
your question you could disallow Delete rights (assuming
NTFS) - Ive never tried it but the option is there.
Adam|||Thanks for the reply - do you think it would still work
well with say 100,000 documents?
Adam

Do you recommend storing images directly into the DB?

The images each are in Jpeg format (black and white, with some grey scale). File size wise, wouldnt be huge < 50 KB.

There wont be too many people using the system.

Would it be tedious to maintain? Many people recommend storing the path to the graphic instead.

Please advise. Thanks.98% of all articles or opinions I've seen reccomend storing them outside the DB. SQL is not a good storage mechanism for files.

HTH|||Originally posted by fused
The images each are in Jpeg format (black and white, with some grey scale). File size wise, wouldnt be huge < 50 KB.

There wont be too many people using the system.

Would it be tedious to maintain? Many people recommend storing the path to the graphic instead.

Please advise. Thanks.

We have a proffesianally built imaging system that stores all the images on the disk and just references them from the database.

I agree that they should be left outside the DB.|||Originally posted by jimpen
We have a proffesianally built imaging system that stores all the images on the disk and just references them from the database.

I agree that they should be left outside the DB.

Right now, we are planning a project to store some images. What's the scripting language that you use? Any advice? Thanks.

So you guys have a file server that has permission given to everyone? And when you upload the pic, the picture is moved to the shared file server? And a path is inserted into the sql server?

Please advise.|||I have an ASP app that does this sort of thing. I read through the directories, and display the file names as hyperlinks without even using a db connection. This method works really well if you can get the user to name the files in such a way that the links on the page look ok...which is normally not an easy task, but it's worked pretty well for me...But this is an IntrAnet app. , not a web app..|||We use ASP.Net with C# on Windows IIS. You know the IP of the computer, create a sub directory (eg /Images) and use a component that looks in this directory (we use Borland C# Builder but Visual Studio will have this probably as well). To let users upload their images, you can use ASPUpload (www.aspupload.com)|||Originally posted by fused
Right now, we are planning a project to store some images. What's the scripting language that you use? Any advice? Thanks.

So you guys have a file server that has permission given to everyone? And when you upload the pic, the picture is moved to the shared file server? And a path is inserted into the sql server?

Please advise.

Ours is provided by a VAR of http://www.imagesoft.com/.

Read permissions are given by group as well as write. We have different sections that are available to different groups depending on what they need/are allowed to see.