November 28, 2009

SQL SERVER SSIS: How to assign Connection from variable

Last Article, SSIS - Foreach Loop Container, We need to assign dynamic connection to file connection, so SSIS For each loop Task can take each file from folder.

Lets configure File connection from variable for SSIS - Foreach Loop Container.

What we need to do is, we need to process each file from folder, so we need to assign value from variable to File connection, so SSIS Task will read that file and process that file.

To assign FileConnection dynamicaly we need to do following.

1. Right click on File Connection, click Properties.

2. Set DelayValidation = "False", as we need to assign connection dynamically.

3. Click on "Expression", and enter variable  name, which we used in SSIS - Foreach Loop Container.

SSIS Foreach Loop Container Connection

That's it.

It will assign connection from variable and process that file.

Let me know if you have any question for the same.

November 9, 2009

SQL SERVER: SSIS - Foreach Loop Container

Today, I am going to explain SSIS For Each Loop Task. I am sure that it will be easy to configure SSIS Package with Foreach Loop task, after you read this article.

The Foreach Loop container defines a repeating control flow in a package. The loop implementation is similar to Foreach looping structure in programming languages. In a package, looping is enabled by using a Foreach enumerator. The Foreach Loop container repeats the control flow for each member of a specified enumerator.
Many times, user asks that I want to process all files from my folder to database. This is my answer to all of those users. SSIS comes up with For Each Loop which did the same.
Now I'm assuming that you're familiar with using the SQL Dev Studio tools and building basic packages. If this isn't the case, I recommend working your way through the Integration Services using my earlier posts.

SSIS Foreach Loop Container

1. SSIS Foreach Loop Container - General : Here we need to assign unique name to this container and also we can specify brief description, so we will get idea why we need to design this container.

SSIS Foreach Loop Container General

2. SSIS Foreach Loop Container - Collection : Use the Collection page of the Foreach Loop Editor dialog box to specify the enumerator type and configure the enumerator.

SSIS Foreach Loop Container Collection

Select the enumerator type from the list. This property has the options listed as follows:
  1. Foreach File Enumerator: Enumerate files
  2. Foreach Item Enumerator: Enumerate values in an item
  3. Foreach ADO Enumerator: Enumerate tables or rows in tables
  4. Foreach ADO.NET Schema Rowset Enumerator: Enumerate a schema
  5. Foreach From Variable Enumerator: Enumerate the value in a variable
  6. Foreach Nodelist Enumerator: Enumerate nodes in an XML document
  7. Foreach SMO Enumerator: Enumerate a SMO object
Let me explain with Foreach File Enumerator

We can use this Foreach File enumerator to enumerate files in a folder.
The content of the folders and subfolders that the Foreach File enumerator enumerates might change while the loop is executing because external processes or tasks in the loop add, rename, or delete files while the loop is executing. This means that a number of unexpected situations may occur:

In Folder option, provide the path of the root folder to enumerate. In Files option, specify the files to enumerate.
If wildcard characters are specified in the Files option, then the fully-qualified paths that are returned match the filter.
  1. Fully qualified: Select to retrieve the fully qualified path of file names.
  2. Name and extension: Select to retrieve the file names and their file name extensions.
  3. Name only: Select to retrieve only the file names.
3. SSIS Foreach Loop Container - Variable Mappings : Use the Variables Mappings page of the Foreach Loop Editor dialog box to map variables to the collection value. The value of the variable is updated with the collection values on each iteration of the loop.
Variable: Select an existing variable, or click <New variable...> to create a new variable.
Index: If using the Foreach Item enumerator, specify the index of the column in the collection value to map to the variable. For other enumerator types, the index is read-only.
Once we configured, For Each Loop container, Lets add the process that we want. Here what I am going to do is:
NOTE: We can add any other tasks to this container. I took Data flow process, as its already explained by this BLOG.

Here lets see the logical scenario of For Each Container Task.

First, From the selected folder, fetch all the files and assign file path to the variable.
Second, For Data Flow Process, process the file from the folder and execute data flow.

NOTE: For Data Flow Process, we need to assign Connection from variable. ( as we assigned variable after each loop, we need to process new file from the folder). So How can we configure File connection to use new file?
Please refer to my next article for How to assign Connection from variable.

That's it. This is what we need to design for "For Each Loop".

Let me know if there is any difficulties you have to design this package.

November 3, 2009

SQL SERVER: Configure Database Mail with SQL SERVER 2005

We have used Database mail to send mail to client on each updates.This is a very simple process to configure. Let me share how to configure Database mail with SQL server 2005 with all of you. After setting up Profile and Account properly, you just need to write following code to send a mail to client:

Step 1:

Configure Database Mail Step 1

Step 2:

Configure Database Mail Step 2

Step 3:

Configure Database Mail Step 3

Step 4: You might get this message:

Configure Database Mail Step 4

Step 5: Create Profile

Configure Database Mail Step 5

Step 6 : Create Account

Configure Database Mail Step 6

That's it.

exec msdb.dbo.sp_send_dbmail
@profile_name = 'ProfileName', IN our CASE, 'Tejas'
@recipients = 'Client Email Address' ,
@blind_copy_recipients = 'BCC Address',
@subject = 'Subject',
@BODY = 'Message Body',
@body_format = 'Message Type', it could be text OR html

Let me know if you have any complexity or comments in setting up Database mail.

November 2, 2009

SQL SERVER SSIS - For Loop Container

Today, I am going to explain SQL SERVER SSIS, For Loop Container.

The For Loop container defines a repeating control flow in a package. The loop implementation is the same concept of the For looping structure in programming languages. In each repeat of the loop, the For Loop container evaluates an expression and repeats its workflow until the expression evaluates to False.

The For Loop container uses the following elements to define the loop:

  • An optional initialization expression that assigns values to the loop counters.
  • An evaluation expression that contains the expression used to test whether the loop should stop or continue.
  • An optional iteration expression that increments or decrements the loop counter.

Let's take an example to easily understand how to use For Loop Container with SSIS. Here I take example to iterate ActiveX Task.

1. Select and Drag, For Loop Container, from Container Flow Items to designer surface and add ActiveX Script Task to run inside the loop.

SSIS For Loop Container

2. To configure this container, right click on this and click on 'edit'.

SSIS For Loop Container editor


First we need to create variables to run this package based on variables. We can create variables by: View -> Other Windows -> Variables. Please find variable screen as below:

For Loop Container 2

Here, I specified both variables. Count and Increment, that I am going to use for this example. I specified value Count = 20. So loop will be executed 20 times.


Let's take a view how each properties are used:

For Loop Properties:

1. InitExpression: Type an Initialization Expression in the given textbox. Initialization ensures that we are starting by setting out increment counter to 1. Here I specified variable to 1.

2.EvalExpression: Type an Evaluation Expression in the  given textbox. For each iteration the evaluation expression checks to see if we have reached our maximum iteration count as set above when we defined @Counter. Here I specified that @Increment <= @Count, code inside the for loop will execute @Count times.

3. AssignExpression: Type an Assignment Expression in the given textbox. This is used to increment the counter by one for each iteration of the loop, otherwise the loop would never finish. Here I specified to increment variable by 1.

That's it. We have configured SSIS For Loop container.

Now when we execute this package, it will execute ActiveX Task, 20 times (as specified in variable count).

Let me know if you have any questions.

October 31, 2009


Today I am going to explain SSIS - ActiveX Task.

Here I am just explain that how to write VB Script in this task, which will execute at run time. This ActiveX Task is mostly used when we  move SQL 2000 DTS package to SSIS. As I found that many developers had used ActiveX with DTS packages in SQL SERVER 2000.

The ActiveX Script task provides a way to continue to use custom code that was developed using ActiveX script, until such scripts can be upgraded to use the more advanced features provided by the Script task.

Let's take an example to easily understand how to use ActiveX Task with SSIS.

1. Select and Drag, ActiveX Script Task, from Container Flow Items to designer surface.
 1 SSIS ActiveX Script Task
2. Now we need to write a script which will execute when SSIS package loads. To write a script, Right click on  ActiveX Script Task, which we dragged to Design surface. Click on "Edit.", you will get page as:

2 SSIS ActiveX Script Task
3. SSIS ActiveX Script Task - General. Here we need to assign unique name to this task and also we can specify brief description, so we will get idea why we design this task.

4. SSIS ActiveX Script Task - Script Task: This is main section where we need to write script.

Language: We need to specify which script language we are going to use for this task, either VB Script, JScript etc..Here we are going to use VB Script language for this demo

Script: Here we need to write a script. Click on ".." button located at right side. It will open a dialog to write a script, as shown in figure:

3 SSIS ActiveX Script Task 
Let's write VB Script code inside to test this task. I write this script:

function Main()

Dim Stuff, myFSO, WriteStuff, dateStamp
dateStamp = Date()

Stuff = "SSIS Text ActiveX Task"

Set myFSO = CreateObject("Scripting.FileSystemObject")
Set WriteStuff=myFSO.OpenTextFile("D:\SQLYoga\SSIS ActiveX Task.txt", 8, True)
SET WriteStuff = NOTHING

End function

Save it and click on Parse, which will parse this script and let us know if there is any problem in this script. If It is on then its allowed to proceed with the next step.

Expression: Use the Expressions page to edit property expressions and to access the Property Expressions Editor and Property Expression Builder dialog boxes.

That's it.

Let's run task, by right click on task and click on Execute Task, as shown in following figure. You can either Execute Package by right click on Package name, from Solution Explorer.

 4 SSIS ActiveX Script Task

Once you run it and if logged in user has write access to the given path, SSIS package will create file at that place. Now go to that given path and see file is created or appended with content that we want to write.

That's simple to configure and use SSIS ActiveX Script Task within BIDS.

Let me know if you have any questions in designing SSIS Package


October 29, 2009


As we have seen SSIS Data Flow TASK in previous article, today I am going to explain SSIS BULK INSERT TASK.

The Bulk Insert task provides an efficient way to copy large amounts of data into a SQL Server table or view. Let's say company is using Online Product Marketing and every day they have to update their databases with the latest information provided by their suppliers in Tab Separated/Comma Separated file. Here BULK INSERT provides efficient way.

We can move data to SQL SERVER Table or View by BULK INSERT Task.

Let see How to configure SSIS BULK INSERT Task:

1. Drag Control: Drag and drop, BULK INSERT TASK from Toolbox to Design surface:

Bulk Insert 1
2. Configure connection: Right click on Task and click on Edit

Specify Destination Connection: Specify the OLE DB connection manager to connect to the destination SQL Server database and the table or view into which data is inserted.

NOTE: The Bulk Insert task supports only OLE DB connections for the destination database.

Define Format:
We need to Define the format that is used by the Bulk Insert task, either by using a format file or by defining the column and row delimiters of the source data. If using a format file, specify the File connection manager to access the format file. We need to specify Row Delimiter and Column Delimiter, as per our requirement.

Specify Source Connection: Finally, we need to specify Flat or File Connection Manager to access the source file. BULK INSERT Task is process this file as per defined format on previous step.

Bulk Insert SSIS Task 2

3. Configure Options: SSIS BULK INSERT Task also provides to specify some options which needs to perform when BULK TASK Insert some data to destination Table/View, like: Check Constraint, Keep Nulls, Enable Identity Insert, Table Lock, Fire Triggers.

Bulk Insert SSIS Task 3

That's it. Now execute the package, you find that your Table/view is populated with data from provided text file.

We can say that Data Flow Task is advanced version of BULK INSERT TASK, which provides more features and more scalable.

The Bulk Insert task can transfer data only from a text file into a SQL Server table or view. To use the Bulk Insert task to transfer data from other database management systems (DBMSs), you must export the data from the source to a text file and then import the data from the text file into a SQL Server table or view.

With Data Flow Task, we can import/export data from any database, we don't need to convert it to text and then import.

So, we can use either of these as per our requirements.

Let me know if you have  any questions.

Reference: SQL SERVER Books Online, Tejas Shah (

October 24, 2009

SQL SERVER SSIS: Basic Example Of Data Flow Task

Today I am going to give Basic example of Data Flow Task in SSIS, import CSV file to SQL SERVER Database

Many times user asked to import their data from their files. They might have data in Microsoft Access, Microsoft Excel, CSV File, Text Tile etc..

Microsoft SQL SERVER, SSIS, has features to import data from any heterogonous format to SQL and also capabilities to export it to any format. That is the power of SSIS.

1. Drag Control: Drag Data Flow Task from Toolbox, to Design interface

Data Flow Task 1
2. Configure Data Flow Task: Once you have Data Flow Task on Design surface, Double click on it, or Right click on task and click on "Edit", it will redirect you to Data Flow Tab, as shown in Figure 2:

Data Flow Task 2 
As shown in above figure, SQL SERVER SSIS give features to import/export data from/to many formats. You can find the different formats that is supported by SSIS to import data on LEFT PANE.

3. Configure Source Connection: Here, I have selected "Flat File Source", as we need to import data from CSV File. Drag "Flat File Source" to design surface and right click on edit and select "Edit", you will get screen like:

 Data Flow Task 3
As shown in figure, Click on "New", It will open a form which allows us to select file which we want to import to our Database.

First, Enter connection manager name and description.
Then, select a file which you want to import to database.
Then, Select Locale and Code page as per your requirement, For this demo, I don't need to change anything.
Then, we need to select Format of the file. SSIS provide us three formats: Delimited, Fixed Width, Ragged Right. We need to import CSV file (Comma Separated file), so I have selected "Delimited".
Then, We need to specify Text Qualifier, Select based upon your requirements.
Then, we need to specify delimiter for Header row, as we have CSV  file, I have selected CR LF, means new row for header.
Then, SSIS also give us features to skip rows. By Header Rows to skip, it allows us to skip n number of rows.
At Last, If there is Header in column names, we just need to select this checkbox, "Column names in the data row". So SSIS understands to set the first row as Header row.

That's it for configure file, Now we need to specify delimiter for column, which comes on next section called "Columns"

 Data Flow Task 4

As shown in figure, Now we need to specify delimiter for Rows and Columns.
As we are going to import CSV file (comma separated file), Row Delimiter is "{CR}{LF}" and column delimiter is ",".

Also notice that, it display Column Names in Header, as we have checked, "Column Names in the First data row".

This is the data that we are going to import to SQL Database.

You can use Advanced and Preview tab, to configure each column and preview, respectively.

Now once, configuration is done, we need to select OLE DB Destination, as we need to import it to SQL SERVER database.

4. Configure Destination Connection: Here, I have selected "OLE DB Destination", as we need to import data from CSV File to SQL SERVER Database. Drag "OLE DB Destination" to design surface.

Data Flow Task 5 
Now, we need to design data flow. Here we need to import data from Flat File connection, I have dragged "Success" (Green Arrow), link to OLE Db Destination for column Mapping.

Now, we need to configure OLE DB destination, to insert data to SQL Database. Right click on it and click on "EDIT", it will open screen like this:

 Data Flow Task 6
Data Flow Task 7

I have entered SQL server Login information to connect to SQL SERVER where I need to import data from CSV file.

Once, SQL SERVER is configured, we need to select Table Name to which we need to insert data from CSV file. See the following screen, I have selected "Employee" Table: Please find script to create "Employee Table"

CREATE TABLE [dbo].[Employee](
    [EmpID] [int] IDENTITY(1,1) NOT NULL,
    [EmployeeName] [varchar](max) NULL,
    [EmployeeCode] [varchar](max) NULL,
    [EmployeeDesignation] [varchar](max) NULL,
    [CreatedDate] [datetime] DEFAULT GETDATE()

Data Flow Task 8

Now we have selected SQL SERVER table to which we need to import data, Now we need to map columns names, so SSIS will insert data accordingly.

Data Flow Task 9

By default, SSIS provides mapping for columns which have same name, In this case Column Mapping is ok, so we don't need to do anything. If you want to change mapping, you can do it by just changing links between Input Columns and Destination Columns.

NOTE: Here Destination table has two more columns, EmpID and CreatedDate, which is not available in source file, CSV file. We can just ignore it, as EmpID is Identity column while CreatedDate is GETDATE(). We have such columns in SQL database to keep track when this record is added.

That's it. We have configured Data Flow Task.

Now just right click on package and "Execute Package". You will find that Employee table will be populated with information from CSV file.

Let me know if you have any questions in designing SSIS Package.

October 22, 2009


SSIS (SQL Server Integration Services) is an upgrade of DTS (Data Transformation Services), which is a feature of the previous version of SQL Server

SSIS (SQL Server Integration Services) is introduced in SQL SERVER 2005. It is totally new transformation of data. Previously we have DTS package in SQL SERVER 2000. Now they introduced SSIS with many features.

SSIS is an extensive data integration platform which is used to transfer and merge the information from various sources and load to multiple systems. It contains wide range of capabilities.

Business Intelligence is used for decision making, future trends based on collected data. I can find very good article written by Microsoft SQL SERVER MVP, Pinalkumar Dave. He write about BI on his one of the best article:

SSIS packages can be created in BIDS (Business Intelligence Development Studio).

I am going to explain all features of SSIS for easy and better understanding by following articles.

Reference: Tejas Shah (

October 8, 2009

Top features of Windows 7 and Office 2010

We have Community TechDays on 3rd October 2009, Saturday at Hotel Anmol, Ahmedabad. This was excellent event, you can find out more details here.

I like these features of Windows 7 and Office 2010 which was presented by Vinod kumar, Microsoft Evangelist.

1. I like the Recorder Tool provided by Windows 7. This is very handy tool which will make our (IT professional) life easy. Many times we find that users complaint that application raise an error. What we need is, what user did so error occurs. By this tool, user can record all of the steps. This tool will capture all screen shots and description what user did. So user just need to send this file to us and we can easily come to know that why this happen. One more thing is that, File size is also not much though all screen shots are there. We can also have option to reduce size, by reducing quality of images.

2. I like the option of Jump List, We will access applications like windows media player just from Task Bar. We don't need to even open that Application. This is very nice feature as everything is just from task bar.

3. There is a feature to Magnify the screen and also point out the location. This we really need when we are giving presentation or in meetings. By this feature we can present things in easy way so all persons can understand easily.

4. Feature: Trigger Start Service. This is really good feature, as previously it was programmed as System keeps check periodically about the new hardware is attached or not. That was really resource consuming. Microsoft resolved this in Windows 7. As Vinod Kumar explained with USB drive at Community TechDays, Event is only fired when we attach any new hardware else it is not in even memory.

5. Now we can also Re-Arrange open applications in Task manager. As we used Tabbed browser and like to move important tabs first. This is the same concept developed in Windows 7 to arrange icons in Task Manager.

6. Show Desktop feature, Windows 7 introduced the "Show Desktop" button at right bottom side. We just need to click on right bottom which will minimize all applications which will  make developers life simple :)

7. Windows 7 also comes with good display options. We can customize the color of Taskbar icon's color.

8. Virtulization, The most common reason is that you want to run Virtual PC on Windows 7 on a system that does not have hardware virtualization support.

9. In Office 2010, feature to notify icon which indicates that recipient is available or not at that time. By this feature, at the time of sending an email come to know that recipient is available or not.

10. Also in Office 2010, Grouping follow up mails. This is feature by which we can find whole list mails, received and sent. So we come to know about the discussion from first email to last email.

11. In Office 2010, You can not copy the copyright contents. I can not get correct name to explain this.

There are many more featured explained by Vinod Kumar @ Community TechDays. Here I just mentioned features those are still on my  mind and I like this in real life.

Reference: Tejas Shah (http://www.SQLYoga.Com)

October 7, 2009

Community TechDays at Ahmedabad - Great Event

There was a great event held by Microsoft and PASS (Professional Association of SQL Server) in Ahmedabad on 3rd October 2009. This was very big event where 250+ attendees attend the excellent session of Vinod Kumar, Pinal Dave, Jacob Sebastian and Prabhjot Singh Bakshi. There were Four technical sessions by these excellent speakers.

First session is of Vinod Kumar. He is Microsoft Evangelist. There was excellent session of Three hours where he give features of Windows 7 and Office 2010.

Second session is of Pinal Dave, SQL MVP. He presented session on the subject - "SQL Server - The Other Side of Index". This was also excellent way to know more about SQL Index. and How to use index. That is really nice session for SQL Developers to know, How to optimize query. It gives proper understanding of indexes.

Pinal Dave
Third session is of Jacob Sebastian, SQL MVP. He presented session on '"The Best Practices for Exception Handling and Defensive Programming in Microsoft SQL Server". This was also very great session where developer can get more idea about When exception occur and how to handle that exception. It was presented in a way so all attendees get clear idea about it and understand it properly.

Jacob Sebastian
Forth and last session is of Prabhjot Singh Bakshi, MCT(Microsoft Certified Trainer). He presented session on "NET Framework 4.0". He also presented very nice session about featured of .Net Framework 4.0. He developed .Net code to give clear understanding of new features.

Prabhjot Singh
Click with Vinod kumar and Jacob Sebastian:

Vinodkumar TejasShah JacobSebastian

Overall, that was excellent day for Ahmedabad. I hope this type of TechDays scheduled in near future.

Reference: Tejas Shah (