Automation with FileMaker Server Scheduler

For many years automating tasks in a database hosted by FileMaker Server required a “robot” computer that used its operating system’s scheduler to open a dummy FileMaker database file at a particular time. The dummy database would then connect to a hosted database, trigger a FileMaker script, and then close FileMaker Pro. Robot machines worked for the most part, but there were many factors that could go wrong and often did. For example, the robot machine had to be on and running properly with an active network connection to the server.

FileMaker Server 9 introduced a feature that allowed FileMaker scripts to be scheduled in the FileMaker Server Admin Console. Scripts could run automatically without having a copy of FileMaker Pro running on the server or on a robot computer. This was a major step toward reliable automation and the end of robot computers.

Unfortunately, because the database wasn’t open in FileMaker Pro, not all script steps could be run on the server. FileMaker Server 10 has taken the script scheduling feature further by allowing server compatibility for many more script steps. Most importantly, FileMaker Server 10 allows for importing and exporting of data on the server. In this article, all examples of automation are based on an environment running FileMaker Pro 10 and FileMaker Server 10.

What would I want to automate?

Over the years IT Solutions has set up automation on many FileMaker systems, and for many different reasons. Most commonly, automation is used to create a set of records in bulk at a particular point in time so that they will be waiting when a user logs in and needs to use them. An example of this kind of automation is timesheet records for employees. On a particular day and time, such as Sunday evening or early Monday morning, timesheet records are created for all active employees. Another example is the creation of attendance records for a school. When a teacher logs in to take attendance, records for the day are already created for the teacher’s current class roster. All he or she has to do is mark the records for absent students.

Another kind of automation we are often asked to create has to do with updating statuses. System dashboards and classroom assignments are examples of statuses within a system that need updated. System dashboards often collect data elements from different sources within the database and then display them in one consolidated graphical overview. Updating dashboards a few times a day is often enough to provide a bird’s eye view of the organization.

In a school setting, students may be assigned to certain classrooms or programs for a particular amount of time. By allowing teachers to enter both start and end dates for those assignments, the system can update a student’s status automatically. Each day a server-scheduled script can run on all classroom assignments and set them to active or inactive based on the dates entered in the system. Additionally, the scripts can add or remove them from the student’s schedule, which may be stored in a separate part of the system.

A third type of automation deals with system integration. Certain types of SQL databases can have a real time connection with FileMaker using the ESS (external SQL source) feature. For systems that are not ESS compatible, integration scripts can be scheduled to run as often as required by an organization’s business rules. For example, when doing integration with an accounting package, such as QuickBooks, it is often required to batch post customer and invoice records at the end, or throughout, each business day. Likewise, balances may need to be pulled from the accounting package back into a FileMaker. These processes can be scheduled to occur as often as needed to allow for accurate representation of data in both systems.

Another example of automation for system integration is importing and exporting data to and from databases to maintain a single authoritative data source for an organization’s collective system. For example, FileMaker cannot establish a real time connection with SAP systems, but FileMaker Server 10 introduced the ability import data from an external file using a scheduled script, so long as it is located in a particular directory on the host server. So, we can schedule the operating system to copy a file into a particular location, and then schedule FileMaker Server to import and update data from that file in a hosted database. This keeps SAP as the authoritative source for certain data, but allows FileMaker users access to use that data at will. All of this can happen in the middle of the night or at particular time intervals without the need for user interaction.

Limitations

How can you tell whether or not a script step will work properly if it is included in a script that is scheduled to run on the server? The Manage Scripts tool in FileMaker Pro 10 offers a convenient compatibility checker located in the bottom left corner of the Edit Script window when modifying or building a script. By default the compatibility checker is set to “Client” which includes all script steps. Changing that value to “Server” or “Web Publishing” shows which script steps are not compatible to each respective environment by changing the color of the script step from black to gray.

Why are some script steps not compatible when running on FileMaker Server? When a script is running on FileMaker Server, no instance of the database will be opened in FileMaker Pro. This prevents certain kinds of actions from being possible. For example, layout dependent script steps, such as adjusting windows, showing a custom dialog box, or entering preview mode, will not be compatible because there is not layout available to the script.

Developers should use caution to only schedule scripts with 100% compatible steps. If the FileMaker Server attempts to run a scheduled script and it encounters an incompatible script step, the script will be aborted and an error will be written to the server log.

If you have any questions about automating your FileMaker solution using FileMaker Server script automation, please do not hesitate to contact me, Jason Mundok, at jason.mundok@itsolutions-inc.com or 866.PICK.ITS.

When FileMaker and QuickBooks integration makes sense

There are two common reasons to choose to integrate FileMaker with QuickBooks, rather than using one or the other exclusively. The first is simply when source data need to be shared between two systems, and the second is when a QuickBooks feature, such as invoicing, needs to be expanded upon and customized to suit the needs of the business.

Sharing data between systems prevents double-entry and the inevitable problems of keeping to separate systems in sync manually. As an example, FileMaker may be used to manage certain parts of a business that QuickBooks simply doesn’t deal with, like a manufacturing process or project management. If some of the same data are used by both systems, such as employee info or vendor records, then either system can be the data source, as long as they can be synchronized often enough to remain accurate.

No matter how robust QuickBooks is, it is still designed for thousands of businesses and therefore cannot completely serve the nuances of individual companies. Many small businesses serve niche markets, offering products and services that are rare or unique, using processes that are different from a typical approach. Often times these niche processes cannot be worked around. Since FileMaker is completely customizable, we can create a system needed to deal with the more complex pieces of the puzzle, and then push the results to QuickBooks in a way that it understands. Then we can let QuickBooks take care of the other pieces of the process that are standard.

A common example of this second reason for integrating QuickBooks and FileMaker is invoicing. If a business needs to invoice in a way that isn’t native to QuickBooks, we can create and produce invoices in FileMaker, roll up and push the invoice data to QuickBooks and allow the remainder of the receivable process to happen there. We can even pull back invoice balances so FileMaker users can check the status of an invoice from inside FileMaker.

How does it work?

Over the years, we’ve used all available means to integrate FileMaker and QuickBooks for our clients. We have found 123sync from SolutionsMakers, Inc. (http://www.solutionmakers.com) to be the easiest and most cost effective way to get the job done. 123sync uses a plug-in in the background to actually push and pull data between systems, but the heavy lifting is already taken care of with a pair of FileMaker “sync” files that are hosted on the same FileMaker server as your solution. First we’ll look at how it works from the FileMaker-to-QuickBooks perspective.

Throughout the work day, records are entered and updated in FileMaker. For example, a user may enter a new customer record, update a customer’s contact information, or create a new invoice for a customer. When completed, these FileMaker records are ready to be sent over to 123sync. This process can either be manual or in a scheduled batch. The IDs are copied into one of 123sync’s FileMaker files and staged. Again, either manually or in a batch at a particular time interval, the records are posted to QuickBooks by 123sync.

Why the extra step by staging the records in 123sync instead of just sending them straight to QuickBooks? 123sync is designed to be the translator between your FileMaker solution and QuickBooks and contains all of the rules necessary for seamless integration. QuickBooks expects data to be posted in a certain order and has many rules to guarantee good data is coming in. 123sync understands those rules so only good data gets into QuickBooks and all records are posted in the correct order. If a FileMaker record breaks one of QuickBooks rules, 123sync logs the error and provides feedback on what went wrong so you fix the problem and guarantee that only good data end up in QuickBooks.

From the other direction, 123sync pulls list data, such as vendors, terms, or shipping methods, from QuickBooks into the FileMaker sync file. Because that data are then stored in FileMaker tables, we can fully integrate with your FileMaker system. We can use the QuickBooks data to drive the value lists in FileMaker to make sure you don’t have to maintain those lists in more than one place. The hardest part is often deciding in which system it makes the most sense to store and maintain the data.

The best thing about 123sync is that the FileMaker sync file is customizable. All of the 123sync processes can be performed manually or automatically as necessary to fit within your business’s workflow. With new features in FileMaker 10, such as script triggers and script timers, we can trigger 123sync functionality from your existing FileMaker solution and let it do all of the heavy lifting in the background. On the other hand, if getting up and running quickly is the priority, the sync file has an easy-to-understand user interface that you can start using right “out of the box”.

If you have any questions about integrating your FileMaker solution with QuickBooks, please do not hesitate to contact me at jason.mundok@itsolutions-inc.com or 866.PICK.ITS

We were at a crossroads since our time entry system is based on Microsoft SQL Server. As a relatively small division of IT Solutions, it wasn’t reasonable for the application development project team to dictate how time is tracked. Fortunately for us, FileMaker 9 introduced a brand new feature called External SQL Sources (ESS), which allows real time connections between a FileMaker system and a separate SQL system.

After making the connection to our SQL- based time tracking database, our project management solution is complete. Employees enter all of their time in one place, whether it’s project development, support work, or internal meetings and initiatives. I have real- time access to those records and can relate a project record in my FileMaker system with the time entry records relevant to that project in SQL Server. I can report project progress at any time based on real time data from both systems.

Here’s a look at how ESS works.

Setting up an ESS connection

Connecting a FileMaker database to an external SQL data source is relatively easy, even if you don’t have a lot of experience working with SQL databases. While technical instructions are outside the scope of this article, I will present some high- level concepts on what it takes to get up and running with ESS.

The most important key is setting up a DSN (data source name) on your computer if your database is local, or on the server if you’re hosting databases using FileMaker Server. The DSN contains all of the information that FileMaker will need to know how to access the external data source, such as the database location and credentials. Setting up the DSN depends on your operating system and the kind of database you would like to connect to. There are detailed instructions for setting up a DSN available in a public technical brief called “Introduction to External SQL Sources”, available at www.filemaker.com. Simply enter “tech brief ess” in the Quick Search field at the top of the main page of the FileMaker web site to access it.

After your DSN is setup, you can create an external data source for it in your FileMaker database. Access External Data Sources from the Manage sub-menu, which is located on FileMaker’s File menu. From there you can add references to other FileMaker databases or to ODBC (external) data sources.

When you add a new reference, selecting ODBC will display the DSN field and the Specify button. This button will display a list of configured DSN entries. After selecting the appropriate DSN and entering a username and password for the external system, FileMaker is ready to use the external data.

Tables from the external source can be added to the relationship graph as table occurrences and accessed like any native FileMaker table. The data source can be accessed by selecting the Data Source drop down at the top of the Specify Table box when adding a new table occurrence to the graph. The ESS tables will then appear in the box.

Working with ESS tables

Because table occurrences on the relationship graph can be based on ESS tables, you can reference them just like any of your native FileMaker tables. You can use them in relationships, base layouts on them, or reference their fields in calculations and scripts. FileMaker will even add the ESS tables to the table list in the Manage Database window. ESS tables are called shadow tables and are listed in italics so you can differentiate between them and tables that are internal. While you cannot add or delete fields in an ESS table, you can add calculation or summary fields to your shadow table in FileMaker, thus extending FileMaker’s powerful calculation engine and reporting functionality to your external data.

What ESS is and is not

While it may be tempting to use FileMaker as a front end to a SQL database , FileMaker, Inc. has made it very clear that the intention of ESS is real- time access to some of the data in your external data sources. Supported external systems that you can connect with include: MS SQL Server 2000, MS SQL Server 2005, MS SQL Server 2008, Oracle 9i, Oracle 10, Oracle 11g, MySQL 5.0 Community Edition, and My SQL 5.1 Community Edition.

This feature was never designed to allow FileMaker to act as a fully functional “front end” for SQL databases. It was, however, designed as a tool to allow FileMaker developers to further integrate their solutions into an organization’s overall data strategy.

Over the past few years, we’ve learned both the power and limitations of ESS. If you have any questions about whether or not ESS is right for your integration challenges, please do not hesitate to contact me at jason.mundok@itsolutions-inc.com or 866.PICK.ITS.