# Event tracking Scheme and Data Acceptance
TA records the user's key behaviors in the product through events, such as registration, login, payment, and event attributes describe the relevant information when the event occurs, such as the registered user's 'source channel', 'operate system' and so on. 'User attributes' describe each user and their latest status, such as the user's gender, grade, cumulative recharge amount.
Before using TA system for analysis, data demander identifies events, event attributes and user attributes to be collected to form data reporting scheme in combination with analysis requirements. Data developer carries out embedment implementation according to the determined embedment scheme and reports data timely, accurately and completely, which is the cornerstone of data analysis.
In the process of practice, common data developers lack tool support, cannot locate data problems in time, and have problems such as missing event tracking data, inconsistent attribute reporting type and planning type, resulting in the reported data cannot be analyzed. The data demander also needs to know the quality of embedded data to ensure the availability of analysis results before analyzing the data.
The data reporting scheme and data acceptance function are designed to solve the common problems in the above data reporting process. The quality of data reporting data is guaranteed by uploading data reporting scheme, setting data processing rules and performing data acceptance function. Functions of each module are described as follows:
- Management data reporting scheme: Upload data reporting scheme to TA platform for unified management and record data reporting changes as the basis for data reporting implementation.
- Set up data processing rules: regard data reporting scheme as data acquisition standard, and set ETL data processing rules to ensure the accuracy of stored data.
- Data acceptance: Obtain the difference between actual data and data reporting scheme, optimize data reporting code continuously and steadily improve data quality.
# I. Manage data reporting scheme
This part includes the operation instructions for adding data reporting scheme, checking data reporting scheme and modifying data reporting scheme.
# 1.1 Add data reporting scheme
Default project manager and supervisors can add data reporting scheme, which includes 3 parts:
(1) data reporting events: record the events to be collected and their attributes
(2) Attributes of public events: Attributes affecting each event can be uploaded as attributes of public events. Attributes of public events need to be set manually during data reporting. See the user manual of each access setting.
(3) User Attributes: Record the user attributes collected
TIP
The data processing rules and data acceptance function do not take effect on the preset attributes. If there is no multi-terminal access requiring attribute alignment, you can choose not to fill in the preset attribute information in the event tracking scheme
- You can choose between [batch upload] or [add one by one] data reporting scheme. When uploading in batches, organize your data reporting scheme according to TA data reporting scheme template format, and you can choose to import it into the data reporting scheme page.
- When a small number of data reportings need to be added in product iterations, the option of [adding] data reportings one by one] can be selected to add event, event attribute or user attribute information based on the current scheme.
If you already have data reported in your project, you want to carry out the maintenance of the data reporting scheme based on the existing design, you can click 'upload' button to upload the data reporting scheme. Note:
- Synchronization of this operation does not include preset attributes
- Since TA cannot identify the 'public event attributes', all event attributes will be synchronized to the 'event', which may be different from your data reporting plan and can be adjusted as needed.
# 1.2 View data reporting scheme
TIP
You can tell whether an event or property has data uploaded by the status ID in front of it. 'Green' means data reported; 'Gray' means unreported.
(1) Click on the name of the event to view the details of the event.
(2) The list can be filtered and viewed by searching for information such as Event Name, Display Name, Event Description, Creator, etc.
(3) Click on View by Attribute to select a non-public event attribute and view the list of events associated with that attribute.
When adding events, you can add tags for events as needed and view them by event tags. Classification tags can be created according to classification conditions such as embedded platform, product modules to which events belong.
# 1.3 Edit data reporting scheme
# 1.3.1 Modify data reporting scheme
After adding data reporting scheme, it can be edited as required. Click the Edit button after the event, common event attributes, user attributes, or click the edit above the detail card to enter the edit status:
(1) Edit rules for events
| Event | Edit status |
|---|---|
| Event name | When the event is not reported, it can be edited; the event has been reported and cannot be edited |
| Display name | Editable |
| Event tags | Editable |
| Event description | Editable |
(2) Edit rules for event attributes
| Event properties | Edit status |
|---|---|
| Attribute name | Not editable |
| Display name | Editable |
| Attribute type | Editable |
| Attribute description | Editable |
WARNING
The same event attribute may be associated with more than one event. If you edit the event attribute display name, attribute type or attribute description on an event detail page, the attribute information under other events will be modified at the same time.
(3) User feature editing rules
| User features | Edit status |
|---|---|
| Attribute name | Not editable |
| Display name | Editable |
| Attribute type | Editable |
| Update method | Editable |
| Attribute tab | Editable |
| Attribute description | Editable |
# 1.3.2 Delete data reporting scheme
(1) You can choose to delete event, event attribute or user attribute under the data reporting scheme. Deleting events or attributes in a data reporting scheme only means deleting them from the data reporting scheme and does not actually delete the reported data.
(2) Click on 'More' in the operation bar to select 'Empty data reporting scheme'. This action clears the contents of data reporting events, public event attributes, and user attributes at the same time and can optionally be uploaded again when cleared.
# II. Set Rules for Data Processing
TA defaults to the attribute type based on the data type when the attribute is first stored. In the actual data reporting process, common problems such as inconsistency between attribute reporting type and expected type lead to the data reported can not support analysis and higher repair costs in the later stage. The function module takes the uploaded data reporting scheme as the data acquisition standard, compares the inbound data with the data reporting information, and sets the data processing rules when there are differences in reporting. This effectively avoids inconsistent data and ensures consistency between reported data and planned data at the beginning of data reporting.
After uploading the data reporting scheme, TA over-manager or administrator can set data processing rules in project management. Choose whether to open the data validation based on embedded point scheme based on the default ETL data processing rules. Settings include:
- Events not in the data reporting scheme → whether it can be stored
- Attributes of events not present in the data reporting scheme ( **i.e. attributes of events not present in all events in the **data reporting scheme) → Whether it can be stored
- Not in the event tracking scheme user feature → whether it can be stored
- Event attributes or user attributes that are inconsistent with the 'attribute type' in the data reporting scheme → whether it can be stored
WARNING
Data processing rules only take effect for newly reported events or attributes that have not yet generated metadata.
For example, a 'recharge' event is not recorded in the event tracking scenario, but the event has been reported. This rule does not take effect for 'recharge' events when you select 'Events not in the data reporting scheme are not stored'.
The default project configuration is 'Free Report' mode, i.e. the data reporting scheme has no effect on ETL data processing rules and can be switched to 'Strong Check' mode or 'Custom' mode. In the strong check mode, the data processing rules are described as follows:
**Event rules ** : events not in the data reporting scheme→ Not stored: that is, you need to maintain the complete data reporting information in the data reporting plan before you can report the data. This rule prevents the repository of dirty data that is not in the plan.
**Attribute rules **: including event attributes/user features that are not in the event tracking scheme → discard attributes, inconsistent with attribute types in the data reporting scheme → discard attributes. This rule prevents invalid attributes from being put into storage and ensures attributes are warehoused with expected attributes by comparing actual attributes with data reporting scheme attributes.
Note: Attribute rules only include custom attributes, not TA preset attributes.
Once set, members can view the data processing rules settings for the current project on the data reporting scheme page:
# III. Data Acceptance
After uploading the data reporting scheme and data reporting, you can carry out data acceptance to obtain the reported data quality. The acceptance result consists of the following two parts:
- All differences between reported data and data reporting plan: whether all events are reported and whether unplanned events (events not in data reporting plan); Whether there are missing attributes, unplanned attributes (not in the data reporting scheme), inconsistent reporting attributes and expected types.
- Null rate for reporting attributes: Support setting null rate threshold which will display exceptions in acceptance results when null rate of attributes is greater than threshold.
WARNING
The data acceptance scope includes events and custom attributes, and does not accept preset attributes.
# 3.1 Data Acceptance
Click 'Data Acceptance' in the upper right corner of the page to configure acceptance rules. You can choose to accept event data or user data.
(1) Acceptance event data
Click to select 'Acceptance Event Data' to set the data range for acceptance. Include event time, event range and filter condition:
- Event time: The time range for the acceptance of the event, which defaults to yesterday and can be customized. Note that this time refers to the #time field in the reported data and does not shift with the project time zone.
- Event range: i.e. events requiring acceptance, default Full Event, can customize the event scope. For data acceptance, select a new version of added event.
- Filter condition: Event data can be filtered. If you collect and report data through a client side SDK, you can specify the data under a certain source for acceptance.
Click 'Next' to set the attribute null rate acceptance rules. Include null rate thresholds and statistical rules:
- Null rate threshold: Set the null rate threshold. When the reported attribute null rate exceeds the threshold, an exception will be displayed in the acceptance result.
- Statistics rule: Choose whether to include empty strings in attribute null rate.
(2) Acceptance of user data
Click to select 'Acceptance User Data' to set the acceptance rules for the acceptance of user data range and attribute null rate. Same as acceptance event data.
# 3.2 Check Acceptance Results
Click on the operation after the acceptance record to view the acceptance result report. The acceptance result of event data includes the following scenarios:
| Acceptance result | Result description |
|---|---|
| Success | The reported data is consistent with the event tracking scheme, and the attribute null rate is lower than the set threshold |
| Not reported | Within the selected acceptance range, there is no data report for the event |
| Abnormal | The following conditions are recorded as an event exception: 1. There are events that are not in the event tracking scheme 2. Exists in the event tracking scheme, but actually has the attribute of reporting 3. There are missing attributes, that is, attributes that are not actually reported in the event tracking scheme 4. There is a report attribute type that is inconsistent with the attribute type in the event tracking scheme 5. The attribute null rate exceeds the set threshold |
The acceptance results of user data include:
| Acceptance result | Results Description |
|---|---|
| Success | The reported attribute data is consistent with the event tracking scheme, and the attribute null rate is lower than the set threshold |
| Abnormal | A user feature exception is recorded when: 1. User features not in event tracking scenarios 2. Attribute missing: that is, in the event tracking scheme, but the actual user feature is not reported 3. The report attribute type is inconsistent with the attribute type in the event tracking scheme 4. The attribute null rate exceeds the set threshold |